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PREFACE 


This  report  presents  the  design,  methods,  and  results  of  a  RAND  study  entitled  "The 
Effects  of  the  DRG-Based  Prospective  Payment  System  on  Quality  of  Care  for  Hospitalized 
Medicare  Patients."  The  study  was  funded  by  the  Health  Care  Financing  Administration. 
This  volume  is  the  second  of  a  series  documenting  the  design,  methods,  and  results  of  a  four- 
year  evaluation  that  began  in  1985.  It  presents  material  published  in  a  peer-reviewed  medi- 
cal journal  (see  below)  and  a  series  of  appendixes  designed  to  supplement  that  material. 

The  study  used  a  time  series  design  to  measure  quality  of  care  in  1981-1982,  before  the 
implementation  of  the  prospective  payment  system  (PPS),  and  in  1985-1986,  after  PPS 
implementation.  Quality  of  care  was  judged  using  a  clinically  detailed  review  of  patients' 
medical  records  and  studying  mortality,  readmission,  and  use  of  nursing  homes  after  hospital 
discharge.  Data  were  collected  for  a  nationally  representative  sample  of  Medicare  patients  at 
least  65  years  of  age  who  were  hospitalized  with  one  of  the  study's  six  study  diseases  or  con- 
ditions: congestive  heart  failure,  acute  myocardial  infarction,  pneumonia,  cerebrovascular 
accident,  hip  fracture,  and  depression. 

Other  publications  from  this  study  as  of  this  date  are  listed  below: 

Kahn,  K.  L.,  R.  H.  Brook,  D.  Draper,  et  al.,  "Interpreting  Hospital  Mortality  Data:  How  Can 
We  Proceed?"  JAMA,  Vol.  260,  No.  24,  pp.  3625-3628, 1988. 

Kahn,  K.  L.,  M.  R.  Chassin,  L.  V.  Rubenstein,  et  al.,  Medical  Record  Abstraction  Form  and 
Guidelines  for  Assessing  Quality  of  Care  for  Hospitalized  Patients  with  Congestive 
Heart  Failure,  RAND,  N-2798-HCFA,  December  1988. 

Kosecoff,  J.,  L.  V.  Rubenstein,  K.  L.  Kahn,  et  al.,  Medical  Record  Abstraction  Form  and 
Guidelines  for  Assessing  Quality  of  Care  for  Hospitalized  Patients  with  Acute  Myocar- 
dial Infarction,  RAND,  N-2799-HCFA,  December  1988. 

Sherwood,  M.  J.,  K.  L.  Kahn,  J.  Kosecoff,  et  al.,  Medical  Record  Abstraction  Form  and  Guide- 
lines for  Assessing  Quality  of  Care  for  Hospitalized  Patients  with  Hip  Fracture,  RAND, 
N-2800-HCFA,  December  1988. 

Roth,  C.  P.,  K.  L.  Kahn,  M.  J.  Sherwood,  et  al.,  Medical  Record  Abstraction  Form  and  Guide- 
lines for  Assessing  Quality  of  Care  for  Hospitalized  Patients  with  Pneumonia,  RAND, 
N-2801-HCFA,  December  1988. 

Rubenstein,  L.  V.,  J.  Kosecoff,  K.  L.  Kahn,  et  al.,  Medical  Record  Abstraction  Form  and 
Guidelines  for  Assessing  Quality  of  Care  for  Hospitalized  Patients  with  Cerebrovascular 
Accident,  RAND,  N-2802-HCFA,  December  1988. 

Wells,  K.  B.,  J.  Kosecoff,  M.  J.  Sherwood,  et  al.,  Medical  Record  Abstraction  Form  and  Guide- 
lines for  Assessing  Quality  of  Care  for  Hospitalized  Patients  with  Depression,  RAND, 
N-2803-HCFA,  December  1988. 

Kahn,  K.  L,  L.  V.  Rubenstein,  M.  J.  Sherwood,  and  R.  H.  Brook,  Structured  Implicit  Review 
for  Physician  Implicit  Measurement  of  Quality  of  Care:  Development  of  the  Form  and 
Guidelines  for  Its  Use,  RAND,  N-3016-HCFA,  September  1989. 

Rubenstein,  L.  V.,  K.  L.  Kahn,  E.  R.  Harrison,  et  al.,  Structured  Implicit  Review  of  the  Medi- 
cal Record:  A  Method  for  Measuring  the  Quality  of  Inhospital  Medical  Care  and  a  Sum- 
mary of  Quality  Changes  Following  Implementation  of  the  Medicare  Prospective  Pay- 
ment System,  RAND,  N-3033-HCFA,  1991. 
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SUMMARY 


INTRODUCTION 

In  an  effort  to  control  rising  health  care  costs,  the  federal  government,  in  1983,  estab- 
lished a  prospective  payment  system  (PPS)  to  reimburse  hospitals  for  inhospital  care  of 
Medicare  patients.  PPS  changed  the  way  Medicare  reimbursed  hospitals  from  a  cost  or 
charge  basis  to  a  prospectively  determined,  fixed-price  system  in  which  hospitals  are  paid 
according  to  the  diagnosis  related  group  (DRG)  into  which  a  patient  is  classified.  The  fixed 
payment  per  patient  provides  financial  incentives  for  hospitals  to  reduce  both  length  of  stay 
and  intensity  of  care,  giving  rise  to  concerns  that  the  quality  of  care  may  have  declined  under 
PPS. 

In  October  1985,  The  RAND  Corporation  began  an  evaluation  of  the  impact  of  the 
DRG-based  PPS  system.  The  project  was  funded  by  the  Health  Care  Financing  Administra- 
tion (HCFA)  and  was  undertaken  in  collaboration  with  the  Professional  Review  Organiza- 
tions (PROs)  in  five  states.  The  objective  of  this  four-year  study  was  to  determine  whether 
PPS  cost-containment  efforts  have  affected  the  quality  of  care  received  by  hospitalized  Medi- 
care patients. 

METHODS 

Six  conditions  were  selected  for  this  evaluation:  congestive  heart  failure,  acute  myocar- 
dial infarction,  hip  fracture,  pneumonia,  cerebrovascular  accident,  and  depression.  These 
conditions  were  selected  because  they  occur  frequently  and,  except  for  hip  fracture  and 
depression,  cause  a  large  number  of  deaths.  In  addition,  well  defined  diagnostic  criteria  exist 
for  these  diseases,  and  their  treatment  was  relatively  stable  from  1981  to  1986.  Finally,  for 
these  diseases,  the  patient's  medical  record  contains  the  information  necessary  to  assess 
patient  sickness  at  the  time  of  hospital  admission  and  the  quality  of  hospital  care.  Hip  frac- 
ture and  depression  were  chosen  to  provide  data  about  care  for  surgical  and  psychiatric  con- 
ditions; these  data  supplement  the  information  provided  by  the  four  medical  conditions.  The 
results  of  the  depression  analyses  will  be  presented  at  a  later  time. 

Five  states  were  selected  for  study,  each  from  a  different  geographic  region  of  the 
nation:  California,  Florida,  Indiana,  Pennsylvania,  and  Texas.  Each  state  was  divided  into 
10  to  12  study  areas,  and  we  gathered  data  from  a  sample  of  four  to  eight  areas  in  each  state, 
for  a  total  of  30  areas. 

To  evaluate  the  impact  of  PPS,  we  used  a  time  series  design  in  which  we  compared 
patients  hospitalized  before  implementation  of  PPS  (January  1981  to  December  1982)  with 
patients  hospitalized  in  the  same  institutions  following  PPS  implementation  (July  1985  to 
June  1986).  We  collected  20  percent  of  the  total  patient  sample  in  1981,  30  percent  in  1982, 
and  50  percent  in  the  post-PPS  time  period. 

Our  sample  is  representative  of  patients  nationally  with  respect  to  hospital  size,  teach- 
ing intensity,  percentage  of  Medicare  admissions,  and  size  of  the  city  in  which  the  hospital  is 
located.  We  oversampled  city/county  hospitals  and  hospitals  with  many  Medicaid  patients  to 
better  estimate  the  effect  of  PPS  on  poor  patients. 
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Study  Sample 

Of  the  305  hospitals  asked  to  participate,  297  actually  did  so.  The  planned  study  sam- 
ple was  17,000  patients.  To  collect  data  on  this  number  of  patients,  21,925  medical  records 
were  selected  for  review.  During  medical  record  review,  5,167  patients  did  not  meet  clinical 
criteria  for  having  the  selected  diseases  (even  though  the  record  was  coded  as  such),  and 
these  were  excluded  from  the  study.  Hospitals  were  unable  to  locate  870  records.  The  final 
study  sample  consisted  of  16,758  patients. 

Measuring  Quality  of  Care 

We  used  both  explicit  and  implicit  measures  to  assess  quality  of  care.  With  explicit 
measurement  each  patient's  care  was  compared  to  predetermined  criteria.  With  implicit 
measurement  each  patient's  medical  record  was  assigned  a  quality  of  care  rating  based  on  a 
physician's  judgment  of  the  adequacy  of  the  care.  A  10  percent  subsample  of  the  entire  study 
sample  of  medical  records  underwent  implicit  review.  The  sample  of  medical  records  receiv- 
ing implicit  review  was  evaluated  by  physician  specialists  from  each  of  the  five  study  states. 

Explicit  Measurement.  We  developed  disease-specific  abstraction  forms  to  collect 
explicit  data  from  the  patient's  medical  record  about  sickness  at  admission,  process  of  care, 
inhospital  outcomes,  and  patient  status  at  discharge. 

To  develop  the  explicit  abstraction  form,  we  reviewed  the  literature  on  quality  of  care 
for  patients  hospitalized  with  each  of  the  six  diseases  and  developed  recommendations  on 
what  information  about  sickness  at  admission  and  processes  and  outcomes  of  care  should  be 
abstracted.  We  then  invited  PRO-selected  physicians  from  the  five  states  to  participate  in  a 
one-day  panel  meeting  to  advise  us  on  which  data  to  collect.  In  addition,  we  sought  advice 
about  whether  the  suggested  information  could  be  found  in  the  medical  records  of  patients 
with  each  disease  in  each  state. 

Following  the  panel  meeting,  we  pilot-tested  the  abstraction  forms  on  medical  records 
from  each  of  the  participating  states,  to  determine  if  items  recommended  for  selection  by  the 
physician  panel  were  actually  contained  in  the  medical  record  and  to  take  note  of  the  dif- 
ferent styles  of  recording  of  information  across  states  and  types  of  hospitals  (i.e.,  university, 
community,  or  rural).  After  analysis  of  the  pilot  study,  we  finalized  six  disease-specific 
abstraction  forms. 

After  completing  a  section  on  exclusion  criteria  to  verify  the  patient's  eligibility  for  the 
study,  data  collectors  used  these  abstraction  forms  to  collect  information  about  each  patient's 
acute  and  chronic,  morbid  and  comorbid  conditions  that  were  present  at  admission.  We  then 
collected  process  data  that  were  applicable  both  to  all  patients  and  to  a  given  patient's  partic- 
ular clinical  situation.  For  example,  we  collected  information  about  the  completeness  of  a 
fever  workup  only  for  patients  with  a  temperature  at  or  above  a  given  level.  We  next  col- 
lected clinical  outcome  data  including  information  on  death,  diagnoses  that  were  made  dur- 
ing the  hospitalization,  and  symptoms,  signs,  or  abnormal  laboratory  values  that  occurred 
during  the  hospitalization.  Finally,  we  obtained  information  from  HCFA  files  on  whether  the 
patient  died,  was  readmitted,  or  resided  in  a  nursing  home  within  30  and  180  days  of  hospi- 
tal admission. 

Implicit  Measurement.  In  developing  disease-specific  instruments  for  implicitly 
measuring  quality  of  care,  we  first  reviewed  the  literature  on  quality  of  care  for  patients  hos- 
pitalized with  each  disease  and  the  literature  on  how  to  do  implicit  review.  We  then  invited 
PRO-selected  physicians  from  the  five  study  states  to  participate  in  six  disease-specific  panel 
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meetings  to  discuss  and  test  implicit  measures  of  quality  of  care.  Physicians  were  asked  to 
detail  the  problems  they  had  previously  experienced  in  implicitly  reviewing  records  for  pur- 
poses of  inhospital  assessment  (e.g.,  morbidity  and  mortality  conferences)  or  PRO  review.  In 
addition,  we  sought  advice  about  the  way  medical  records  might  differ  by  type  of  hospital  and 
geographic  area  so  that  the  implicit  review  form  for  measuring  quality  of  care  could  be 
developed  for  use  in  all  kinds  of  acute  care  general  hospitals  in  the  United  States. 

From  this  work,  we  developed  disease-specific  Structured  Implicit  Review  forms  to 
guide  physicians  as  they  used  the  medical  records  to  implicitly  assess  quality  of  care.  For 
each  item  in  each  disease  we  developed  guidelines  to  encourage  consistency  in  the  way  the 
items  were  interpreted  by  physician  reviewers.  Physicians  were  asked  to  review  medical 
records  and  to  use  their  judgment  to  assess  sickness  at  admission,  processes  of  care,  status  at 
discharge,  and  inhospital  outcomes.  Our  intent  was  to  increase  the  reliability  of  implicit  phy- 
sician quality  of  care  review  while  preserving  the  subtlety  of  physician  judgment. 

The  Structured  Implicit  Review  form  specifies  each  part  of  hospital  care  to  be  judged 
(e.g.,  initial  data  gathering,  technology  use,  and  medication  use),  and  provides  guidelines  for 
measuring  care  in  each  area.  The  basic  principle  underlying  the  guidelines  is  that  adequate 
care  in  the  United  States  is  care  that  minimizes  the  risk  of  complications,  maximizes  the 
likelihood  of  a  good  outcome,  and  maximizes  humane  treatment  of  the  patient  at  a  level 
achievable  by  motivated  practitioners  under  average  conditions  at  average  U.S.  hospitals. 
Physician  reviewers  were  asked  to  avoid  adjusting  ratings  according  to  guesses  about  the 
size  or  type  of  hospital  in  which  a  patient  was  treated. 

Data  Collector  Training,  Abstraction,  and  Monitoring 

An  initial  group  of  nurses  and  medical  records  personnel  experienced  with  the  gather- 
ing of  clinical  data  was  selected  by  the  PROs  to  perform  explicit  data  collection.  Using  medi- 
cal records,  the  group  was  then  screened  for  abstraction  skills.  After  demonstrating 
adequate  skills,  52  data  collectors  were  invited  to  participate  in  a  14-day  intensive  training 
session  at  RAND.  Data  collectors  were  taught  how  to  identify  the  proper  medical  record  for 
abstraction  and  how  to  abstract  data  from  the  record.  They  were  also  trained  in  the  collec- 
tion of  additional  data,  such  as  adding  supplemental  clinical  information  verbatim  in  the 
margins  of  the  abstraction  forms  and  attaching  photocopies  of  specified  pieces  of  data  to  the 
forms  (e.g.,  chest  X-ray  reports,  admission  histories  and  physicals,  and  discharge  sum- 
maries). 

During  the  period  of  data  collection,  abstractors  were  monitored  and  feedback  was  given 
to  data  collectors  who  were  identified  as  having  problems.  Additional  fieldwork  was  super- 
vised by  a  project  manager  who  was  familiar  with  the  hospital  in  which  the  abstraction  took 
place.  In  addition,  each  completed  abstraction  form  was  reviewed  by  both  a  physician  and  a 
nonphysician  to  assess  internal  consistency  and  to  verify  that  the  data  collector's  coding  was 
consistent  with  the  supporting  clinical  data  attached  to  the  form.  Records  with  discrepancies 
that  could  not  be  resolved  during  the  review  process  were  returned  for  reabstraction. 

During  the  period  of  data  collection,  the  research  team  conducted  supplementary  train- 
ing sessions  in  the  field,  monitored  the  quality  of  data  collection  through  regular  field  visits, 
scheduled  phone  calls,  and  staffed  a  physician  telephone  hotline. 
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Analysis 

For  each  disease,  we  studied  outcomes  before  and  after  adjustment  for  patient  sickness 
at  admission.  Specifically,  we  measured  mortality  inhospital,  30  days  after  admission,  and 
180  days  after  admission,  as  well  as  inhospital  nonfatal  complications.  In  addition,  we  mea- 
sured length  of  stay,  discharge  destination,  nursing  home  stay  six  months  after  discharge, 
and  readmission  rates  pre-  and  post-PPS. 

To  develop  disease-specific  sickness-at-admission  scales,  we  used  logistic  regression  of 
death  30  and  180  days  after  admission  on  sickness-at-admission  variables.  We  collected  an 
average  of  73  sickness  variables  per  disease,  but  our  final  sickness-at-admission  scales 
predicting  mortality  within  30  days  of  admission  are,  on  average,  weighted  sums  of  19  vari- 
ables. These  scales  explain  about  25  percent  of  the  variance  in  mortality  for  patients  with 
acute  myocardial  infarction,  pneumonia,  or  cerebrovascular  accident. 

We  used  clinical  judgment  and  Likert  scales  to  construct  five  detailed  explicit  process 
scales  and  one  overall  process  scale.  The  scales  measured  physician  cognitive  diagnostic  pro- 
cess, nurse  cognitive  process,  technical  diagnostic  process,  technical  therapeutic  process, 
monitoring  with  telemetry,  and  the  intensive  care  unit.  Compliance  was  high  for  most  of  the 
approximately  100  criteria  per  disease,  but  significant  differences  in  mortality  were  apparent 
for  patients  receiving  good  as  compared  with  poor  process  of  care.  The  positive  relationship 
between  measures  of  process  and  outcome  helped  to  validate  our  explicit  process  scales.  We 
also  found  a  positive  relationship  between  process  and  outcomes  with  our  implicit  measures. 
In  addition,  we  demonstrated  a  strong  relationship  between  our  explicit  measures  of  process 
of  care  and  quality  of  care  as  measured  by  implicit  review  of  the  medical  record  by  physi- 
cians. 

To  summarize  the  changes  in  quality  of  care  associated  with  the  implementation  of 
PPS,  we  compared  sickness  at  admission,  process  (both  implicit  and  explicit),  and  outcomes 
scales  before  and  after  PPS  implementation.  When  appropriate  we  adjusted  for  changes  in 
sickness  at  admission.  For  example,  in  making  outcome  comparisons  pre-  and  post-PPS  it  is 
necessary  to  adjust  for  admission  sickness,  so  that  observed  outcome  differences  are  not 
attributable  simply  to  changes  in  how  sick  patients  are  when  they  arrive  at  the  hospital. 

To  strengthen  our  pre-  post-PPS  comparisons,  we  supplemented  them  with  an  estimate 
of  the  impact  on  mortality  of  each  of  the  described  changes  associated  with  PPS,  with  trend 
analyses  to  see  if  post-PPS  values  were  consistent  with  pre-PPS  trends,  and  with  patient 
subset  analyses,  to  see  if  post-PPS  changes  were  consistent  across  patient  subgroups. 

Confidentiality 

To  maintain  confidentiality  of  hospitals  and  patients  for  the  explicit  data,  coded 
identifiers  were  assigned  to  hospitals  and  patients.  No  identifying  information  about  physi- 
cians was  obtained.  For  the  implicit  review,  all  medical  records  were  de-identified  with 
respect  to  hospital,  physician,  and  patient  name  before  physician  review. 

RESULTS 

This  report  presents  results  for  five  of  the  six  diseases  examined.  The  analysis  of 
depression  has  not  yet  concluded  and  the  results  for  that  disease  will  be  reported  elsewhere. 
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Sickness  at  Admission  and  Length  of  Stay 

Following  PPS  implementation,  patients  are  sicker  at  the  time  of  hospitalization.  We 
estimate  that,  if  everything  else  had  remained  the  same,  the  mortality  rate  30  days  after 
admission  for  our  five  diseases  combined  would  have  been  1.0  percentage  points  higher  in  the 
post-PPS  period  than  its  value  (15.4  percent)  in  the  pre-PPS  period,  just  from  the  rise  in 
admission  sickness.  The  rise  in  sickness  at  admission  was  observed  in  all  five  diseases  and 
was  statistically  significant  for  patients  with  pneumonia  and  hip  fracture  and  for  all  five 
diseases  combined. 

Length  of  stay  dropped  significantly  for  each  of  the  study  diseases.  Hospitalizations 
were  shorter  by  3.4  days  on  average — a  24  percent  decline  across  the  five  diseases  combined. 

Process  of  Care  and  Stability  at  Discharge 

For  most  process  measures,  better  process  was  associated  with  better  inhospital  out- 
comes. As  process  changes  from  good  to  poor,  the  risk  of  inhospital  death  increases 
significantly,  with  relative  risks  for  death  within  30  days  ranging  up  to  1.7,  depending  on  the 
condition  and  type  of  process  measure  examined. 

For  all  of  our  process  measures,  process  of  care  improved  after  PPS  implementation. 
This  is  true  for  physician  cognitive  assessment,  nurse  assessment,  use  of  diagnostic  and 
therapeutic  technologies,  and  for  monitoring  with  intensive  care  and  telemetry. 

Patients  were  sicker  or  less  stable  at  discharge  during  the  post-PPS  period  than  they 
were  in  the  pre-PPS  period.  Instability  at  discharge  is  also  strongly  related  to  postdischarge 
mortality.  Patients  discharged  with  at  least  one  instability  are  1.6  times  more  likely  to  die 
within  90  days  after  discharge  than  patients  discharged  without  instability.  During  the  pre- 
PPS  period,  15  percent  of  patients  were  discharged  with  instability;  post-PPS  this  figure  is  3 
percentage  points  (20  percent)  higher.  When  discharge  instability  and  sickness  are  combined 
with  other  related  factors  such  as  discharge  to  a  nursing  home,  total  discharge  problems 
have  increased  enough  post-PPS  to  raise  death  rates  180  days  from  discharge  0.2  to  0.9  per- 
centage points,  depending  on  disease. 

Patient  Outcomes 

Mortality.  After  adjustment  for  changes  in  sickness  at  the  time  of  hospital  admission, 
we  found  a  reduction  in  inhospital  mortality  for  all  five  diseases  after  the  introduction  of  pro- 
spective payment.  Adjusted  inhospital  mortality  for  the  five  diseases  combined  dropped  from 
16.1  percent  to  12.8  percent.  The  3.3  percentage  point  reduction  in  inhospital  mortality  (95 
percent  confidence  interval  2.3  percent  to  4.3  percent)  was  significant  (p  <  0.01),  as  was  the 
reduction  in  mortality  for  four  of  the  five  diseases. 

When  we  studied  mortality  in  a  fixed  time  period  after  hospital  admission,  adjusted 
mortality  rates  for  all  five  diseases  remained  lower  post-PPS  compared  with  pre-PPS.  How- 
ever, the  relationship  was  no  longer  statistically  significant  at  the  0.01  level.  Unweighted 
adjusted  mortality  rates  30  days  after  admission  for  the  five  diseases  combined  were  16.7 
percent  pre-PPS  and  15.7  percent  post-PPS,  a  difference  of  1.0  percentage  points  (95  percent 
confidence  interval  -0.1  percent  to  2.1  percent,  p  =  0.07).  After  reweighting  the  sample  to 
represent  the  nation,  the  30-day  postadmission  adjusted  mortality  rates  were  slightly  dif- 
ferent from  the  unweighted  values  (16.5  percent  pre-PPS  compared  with  15.4  percent  post- 
PPS,  difference  1.1  percentage  points,  p  =  0.04). 
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The  adjusted  mortality  rates  180  days  after  admission  for  the  five  diseases  combined 
were  29.6  percent  pre-PPS  and  29.2  percent  post-PPS.  Thus,  almost  one-third  of  Medicare 
patients  hospitalized  with  the  five  study  diseases  expired  within  six  months  after  admission. 
For  congestive  heart  failure,  cerebrovascular  accident,  and  hip  fracture,  mortality  dropped 
post-PPS  (significantly  for  hip  fracture:  17.8  percent  pre-PPS,  14.8  percent  post-PPS,  p  < 
0.05),  whereas  for  acute  myocardial  infarction  and  pneumonia,  mortality  rose  post-PPS. 

Discharge  Destination.  For  the  five  diseases  combined,  the  proportion  of  patients 
with  a  preadmission  residence  of  home  and  a  discharge  destination  of  home  was  77  percent 
pre-PPS  and  73  percent  post-PPS  (p  <  0.05),  with  the  most  important  difference  being  for  hip 
fracture  patients  (56  percent  pre-PPS  and  48  percent  post-PPS,  p  <  0.05).  We  found  little 
change  in  the  discharge  destination  for  patients  admitted  from  a  nursing  home.  Overall,  95 
percent  of  all  patients  admitted  from  a  nursing  home  who  also  survived  hospitalization 
returned  to  a  nursing  home,  and  this  did  not  vary  significantly  by  disease  or  by  time  period. 

Prolonged  Nursing  Home  Stay.  In  studying  prolonged  nursing  home  stay,  we 
identified  patients  in  three  of  our  five  states  whose  preadmission  residence  was  home  and 
who  were  still  alive  seven  months  after  hospitalization.  For  the  five  diseases  combined,  8 
percent  of  such  patients  in  two  states  were  in  some  type  of  nursing  home  (skilled  or  residen- 
tial) approximately  six  months  after  the  hospitalization,  whereas  2  percent  of  such  patients 
in  the  third  state  were  in  a  skilled  nursing  home. 

Across  the  three  states  we  found  that  5.6  percent  of  patients  in  the  pre-PPS  period  and 
6.6  percent  of  patients  in  the  post-PPS  period  were  in  nursing  homes  six  months  following 
hospitalization  (p  >  0.05).  This  finding  was  not  statistically  significant  in  any  state. 

Readmissions.  We  found  that  the  number  of  patients  who  died  or  who  had  at  least 
one  readmission  within  180  days  of  admission,  for  the  five  diseases  combined,  was  unchanged 
during  the  two  time  periods:  57  percent  pre-PPS  and  56  percent  post-PPS  (p  >  0.05).  Results 
varied  slightly  by  disease,  with  fewer  congestive  heart  failure,  pneumonia,  and  hip  fracture 
patients  post-PPS  either  dying  or  being  readmitted  (p  >  0.05)  but  more  patients  with  acute 
myocardial  infarction  suffering  one  of  these  two  outcomes  post-PPS  (p  <  0.05). 

The  proportion  of  those  patients  discharged  alive  within  one  year  of  admission  who  had 
at  least  one  hospital  readmission  was  two  percentage  points  lower  post-PPS  for  all  diseases 
combined  and  was  lower  for  each  disease  individually  except  acute  myocardial  infarction  (p  < 
0.05  for  congestive  heart  failure  and  hip  fracture;  p  >  0.05  for  the  other  diseases  individually 
and  for  the  five  diseases  combined).  Across  all  diseases  except  acute  myocardial  infarction, 
the  total  number  of  days  spent  in  the  hospital  within  one  year  of  the  study  hospitalization 
was  two  days  less  post-PPS  than  pre-PPS  (p  <  0.05  for  congestive  heart  failure,  pneumonia, 
hip  fracture,  and  the  five  diseases  combined). 

Summary  of  Outcomes  Pre-  and  Post-PPS.  For  the  five  diseases  combined,  inhospi- 
tal  mortality  was  three  percentage  points  lower  post-PPS  than  pre-PPS  (p  <  0.01).  This 
post-PPS  improvement  in  mortality  decreased  to  1.0  percentage  points  by  30  days  postadmis- 
sion  and  was  almost  gone  by  180  days  postadmission. 

For  patients  admitted  to  the  hospital  from  home,  for  the  five  diseases  combined,  we 
found  that  4  percent  more  patients  discharged  alive  post-PPS  were  not  discharged  to  home 
(p  <  0.05).  We  found  that  1  percent  (0-3  percent  depending  on  the  state)  more  patients  with 
a  preadmission  residence  of  home  were  in  a  nursing  home  six  months  following  hospitaliza- 
tion post-PPS  (p  >  0.05).  For  the  five  diseases  combined  the  proportion  of  patients  with  one 
or  more  readmissions  within  one  year  of  the  initial  hospitalization  was  3.0  percentage  points 
lower  post-PPS  than  pre-PPS  (p  >  0.05). 
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Effects  of  PPS  Implementation 

Post-PPS  improvements  in  inhospital  process  of  care  (after  accounting  for  changes  in 
demographics,  sickness  at  admission,  and  initial  "do  not  resuscitate"  orders)  were  associated 
with  noticeable  decreases  in  expected  mortality  for  all  five  diseases  (0.5  to  1.3  percentage 
points).  Trend  and  patient  analyses  support  the  view  that  improvement  in  process  was  due 
to  ongoing  trends  in  medicine  rather  than  to  PPS. 

In  contrast,  post-PPS  increases  in  problems  at  discharge  have  risen  in  a  way  that  seems 
associated  with  the  implementation  of  PPS.  These  discharge  problems  have  increased 
enough  post-PPS  to  raise  expected  death  rates  by  an  amount  that  varies  from  a  total  of  0.2 
percentage  points  for  acute  myocardial  infarction  patients  to  0.9  percentage  points  for  pneu- 
monia patients.  Furthermore,  instability  at  discharge  is  increasing  across  both  patient  and 
hospital  subgroups.  Both  results  implicate  PPS. 

DISCUSSION 

Before  discussing  the  relationship  between  the  introduction  of  PPS  and  changes  in  med- 
ical outcomes  following  hospitalization,  it  is  important  to  place  these  results  in  context  by 
noting  the  severe  burden  of  illness  carried  by  elderly  patients  hospitalized  with  one  of  the 
five  conditions  examined  in  this  study.  Within  one  month  of  admission,  16  percent  of  these 
patients  have  expired,  with  the  death  rate  climbing  to  29  percent  within  six  months  of  admis- 
sion. For  our  surgical  condition — hip  fracture — the  six-month  mortality  rate  is  16  percent, 
but  for  the  other  four  medical  conditions  it  is  over  33  percent. 

This  burden  of  illness  is  also  demonstrated  by  readmission  rates.  Of  those  patients  who 
survived  the  initial  hospitalization,  more  than  half  were  readmitted  in  the  year  following 
their  hospitalization.  This  fraction  is  highest  for  patients  initially  hospitalized  with  conges- 
tive heart  failure  (66  percent)  and  lowest  for  hip  fracture  patients  (44  percent).  In  addition, 
across  all  our  conditions,  25  percent  of  patients  admitted  from  home  and  discharged  alive  are 
discharged  to  an  institution.  The  corresponding  figures  for  cerebrovascular  accident  and  hip 
fracture  patients  are  41  percent  and  52  percent,  respectively. 

Before  and  after  the  implementation  of  PPS,  clinicians,  patients,  and  families  have 
feared,  and  in  some  instances  have  reported,  disasters  in  outcomes  of  care  that  were  thought 
to  be  related  to  the  new  financial  incentives.  We  have  measured  outcomes  pre-  and  post-PPS 
on  a  nationally  representative  sample  of  more  than  14,000  patients  who  were  hospitalized 
with  one  of  five  diseases  that  constitute  19  percent  of  Medicare  admissions  and  32  percent  of 
deaths  within  30  days  of  admission.  In  contrast  to  these  fears  and  anecdotal  reports,  we  find 
no  significant  increase  post-PPS  in  mortality  at  30  and  180  days  posthospital  admission,  nor 
any  significant  increase  in  readmissions  or  prolonged  nursing  home  stay.  In  fact,  inhospital 
processes  of  care  were  noticeably  better  in  1985-1986  than  in  1981-1982.  We  did  find  a 
significant  increase  in  the  fraction  of  patients  discharged  directly  to  an  institution,  but  this 
does  not  appear  to  have  resulted  in  a  significant  increase  in  nursing  home  stay  at  six  months 
posthospitalization. 

The  main  adverse  change  associated  with  PPS  is  a  20  percent  rise  in  the  rate  with 
which  patients  are  discharged  from  the  hospital  with  instability.  Total  discharge  problems, 
including  instability  and  sickness  at  discharge  and  discharge  to  a  nursing  home,  have 
increased  enough  post-PPS  to  raise  death  rates  0.2  to  0.9  percentage  points. 
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Four  caveats  are  worth  bearing  in  mind  when  considering  our  results:  (1)  We  studied 
only  five  diseases,  albeit  five  important  diseases  in  the  Medicare  cohort.  (2)  Our  study  design 
only  allowed  us  to  assess  differences  in  quality  of  care  once  patients  are  hospitalized,  so  that, 
for  example,  we  cannot  comment  on  any  changes  in  access  to  hospital  care  that  PPS  may 
have  caused.  (3)  Our  post-PPS  payment  data  came  from  1985-1986 — a  period  in  which  the 
mechanism  for  determining  the  hospital's  payment  was  still  changing.  The  rates  paid  to  hos- 
pitals at  that  time  included  a  component  based  on  historical  charges  during  the  cost-plus 
reimbursement  era.  In  addition,  the  amount  of  reimbursement  for  a  patient  with  a  given 
diagnosis  has  changed  since  1986,  and  hospital  financial  margins  have  diminished.  (4) 
Although  outcomes  other  than  discharge  instability  have  not  worsened  on  average,  some  sub- 
groups of  patients  may  have  suffered.  Analyses  to  address  this  concern  are  ongoing. 

Even  with  these  caveats,  two  key  policy  conclusions  appear  clear:  (1)  At  least  through 
the  middle  of  1986,  PPS  did  not  interrupt  an  important  long-term  trend  toward  better 
processes  of  inhospital  care — a  trend  that  has  led  to  somewhat  lower  death  rates.  (2)  On  the 
other  hand,  we  believe  that  PPS  has  had  a  detrimental  effect  on  patients'  stability  at 
discharge.  These  conclusions  lead  to  two  major  policy  recommendations:  (1)  To  eliminate 
any  existing  problems  with  patient  instability  at  discharge,  physicians,  hospitals,  and  PROs 
should  undertake  a  more  systematic  assessment  of  the  readiness  of  a  patient  to  leave  the 
hospital,  so  that  patients  are  not  discharged  to  home  in  a  clinically  unstable  state.  (2)  To  pro- 
vide current  information  about  the  effects  of  Medicare's  payment  methods  on  quality  of  care, 
clinically  detailed  data  that  monitor  sickness  at  admission,  processes,  discharge  status,  and 
outcomes  should  continue  to  be  collected  on  a  regular  basis  as  long  as  prospective  payment  is 
in  place. 
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I.  INTRODUCTION 
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The  health  care  costs  of  the  Medicare  program  to  the  U.S.  government  have  risen  sub- 
stantially over  the  last  25  years.  Under  the  retrospective  payment  system  in  effect  during 
Medicare's  first  two  decades,  costs  rose  at  a  rate  much  higher  than  background  inflation. 
Between  1967  and  1984,  for  instance,  Medicare  expenditures  for  inpatient  hospital  care  rose 
from  about  $3  billion  to  about  $40  billion,  and  the  growth  was  particularly  rapid  in  the  early 
1980s.  Medicare  payments  for  hospital  services  increased  19  percent  annually  from  1979  to 
1981,  and  hospital  charges  rose  almost  16  percent  in  1982,  at  a  time  when  inflation  was  at 
about  5  percent  (ProPac,  1986;  DHHS,  1982). 

In  1982,  after  review  of  the  financial  incentives  associated  with  the  retrospective  pay- 
ment system,  Congress  passed  the  Tax  Equity  and  Fiscal  Responsibility  Act  (TEFRA),  which 
placed  a  prospective  cap,  beginning  in  October  1982,  on  annual  operating  revenues  per  inpa- 
tient Medicare  case  at  each  hospital.  This  was  followed  in  October  1983  by  the  introduction 
of  the  present  prospective  payment  system  (PPS),  in  which  hospitals  are  paid  an  amount 
based  largely  on  flat  rates  per  admission  calculated  for  each  of  approximately  470  diagnosis 
related  groups  (DRGs).  At  the  same  time,  the  current  system  of  professional  review  organi- 
zation (PRO)  monitoring  of  quality  and  appropriateness  of  care  was  established. 

The  new  payment  system  has  been  somewhat  successful  at  slowing  the  upward  spiral  of 
Medicare  costs  (Guterman  et  al.,  1988).  However,  because  prospective  payment  contains 
incentives  to  decrease  lengths  of  stay  and  to  substitute  lower-cost  services  and  procedures, 
patients,  physicians,  and  policymakers  are  concerned  that,  despite  the  introduction  of  PRO 
monitoring,  the  quality  of  health  care  offered  to  Medicare  patients  may  have  declined  under 
PPS.  Since  1985  we  have  been  conducting  a  large  national  study,  funded  by  the  Health  Care 
Financing  Administration  (HCFA)  of  the  Department  of  Health  and  Human  Services,  to 
investigate  the  effects  of  PPS  on  quality  of  care  for  hospitalized  Medicare  patients. 

The  goals  of  our  study  were  (1)  to  assess  the  quality  of  inhospital  care,  for  Medicare 
patients  age  65  and  over,  before  and  after  the  implementation  of  PPS,  and  (2)  to  estimate  the 
effects  of  the  PPS  intervention  on  quality  of  care,  by  comparing  quality  of  care  now  with  our 
best  estimate  of  what  it  would  have  been  in  the  absence  of  PPS.  The  remaining  sections  of 
this  report  present  our  findings.  In  this  introductory  section  we  provide  an  overview  of  our 
methods,  offer  a  review  of  the  recent  literature  on  the  effects  of  PPS,  comment  on  the  gen- 
eralizability  of  our  findings,  mention  some  caveats  in  interpreting  our  results,  and  discuss  a 
key  design  choice  we  faced  and  its  implications  for  future  data  collection  in  quality  of  care 
assessment. 
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OVERVIEW  OF  OUR  METHODS 

We  were  not  able  to  conduct  a  prospective  controlled  trial  of  the  effects  of  PPS  on  qual- 
ity of  care,  since  virtually  the  entire  country  was  put  on  prospective  payment  at  about  the 
same  time.  Instead,  we  designed  a  retrospective  before-after  study,  in  which  we  contrasted 
data  on  Medicare  patients  before  and  after  1983.  We  selected  calendar  years  1981  and  1982 
as  our  pre-PPS  study  period  and  July  1985  to  June  1986  as  our  post-PPS  period,  concentrat- 
ing half  our  sampling  in  each  of  these  periods. 

To  measure  quality  of  care  by  looking  for  differences  in  quality  of  care  across  time 
periods  or  hospitals,  we  naturally  examined  patient  outcomes  such  as  mortality  and  hospital 
readmission  rates.  But  patient  sickness  at  admission  also  needs  to  be  measured  and 
adjusted  for  in  making  outcome  comparisons,  because  patients'  greater  sickness  would  be 
enough  to  explain  moderate  differences  in  outcome  even  if  nothing  else  changed.  A  third 
component — process  (what  clinicians  do  to  patients) — is  also  important:  If  outcomes  adjusted 
for  sickness  at  admission  change  and  we  do  not  examine  the  processes  of  care,  we  cannot 
know  why  the  change  occurred  and  which  aspects  of  care  are  now  better  and  which  are 
worse.  Finally,  because  of  PPS  incentives  to  reduce  lengths  of  stay,  we  also  measured  patient 
instability  at  discharge.  We  compared  outcomes  pre-  and  post-PPS  after  adjusting  for  sick- 
ness at  admission;  we  measured  the  appropriateness  of  the  processes  of  care  given,  through  a 
series  of  explicit  process  criteria  and  through  implicit  review  by  expert  clinicians;  and  we 
tried  to  establish  process-outcome  links,  in  which  better  care  is  shown  to  be  associated  with 
better  outcomes,  that  would  help  explain  any  outcome  differences  we  found.  We  also  studied 
the  relationship  between  sickness  and  instability  at  discharge  and  subsequent  mortality. 

To  gather  clinically  detailed  data  on  sickness  at  admission,  inhospital  processes  and 
outcomes  of  care,  and  status  at  discharge,  we  used  a  stratified,  multistage  cluster  sampling 
plan  that  selected  16,758  patients  from  297  hospitals  in  30  areas  in  five  states.  We  linked 
these  data  with  postdischarge  outcome  information  on  mortality,  nursing  home  stays,  and 
hospital  readmissions  drawn  from  files  maintained  by  HCFA  and  fiscal  intermediaries  in  our 
chosen  states.  The  final  sample  was  nationally  representative  with  respect  to  six  hospital- 
level  factors — size,  teaching  status,  ownership,  percentages  of  Medicare  and  Medicaid 
patients,  and  size  of  the  city  in  which  the  hospital  is  located — as  well  as  patient  demograph- 
ics and  mortality. 

We  based  our  evaluation  of  changes  in  quality  of  care  on  six  diseases  or  conditions: 
congestive  heart  failure,  acute  myocardial  infarction,  pneumonia,  cerebrovascular  accident, 
hip  fracture,  and  depression.  We  review  below  the  logic  behind  our  choice  of  these  conditions 
and  the  implications  of  the  selection  process  for  the  generaliz ability  of  our  findings.  In  this 
report,  however,  we  present  results  only  from  the  first  five  of  these  conditions. 

To  identify  comparable  patients  pre-  and  post-PPS,  we  defined  each  disease  by  ICD-9- 
CM  codes  so  that  any  patient  with  one  of  these  six  conditions  in  either  period  as  the  principal 
reason  for  admission  should  have  had  one  of  these  codes,  even  if  there  were  coding  changes 
over  time.  We  then  used  strict  clinically  detailed  inclusion  criteria  to  select  our  sample  of 
patients,  excluding  24  percent  of  the  records  we  examined  in  an  attempt  to  achieve  homo- 
geneity within  each  disease  or  condition  category. 

We  monitored  the  fieldwork  closely  to  insure  the  accuracy  of  the  data  collection  process. 
For  explicit  review,  PROs  in  our  chosen  states  selected  nurses  and  medical  records  personnel 
experienced  with  clinical  data  to  become  data  collectors;  these  persons  underwent  17  days  of 
training  in  the  use  of  the  disease-specific  abstraction  forms  we  designed  specifically  for  the 
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study.  Each  abstraction  form  contained  approximately  700  items  and  took  an  average  of  90 
minutes  to  complete.  We  conducted  interrater  reliability  studies,  maintained  a  telephone 
hotline  to  resolve  abstraction  difficulties  as  they  occurred,  and  had  physicians  and  nurses 
review  the  completed  abstraction  forms  for  internal  consistency.  As  a  result  of  these  moni- 
toring efforts,  83  percent  of  our  item-level  kappa  scores  were  at  or  above  levels  generally 
recognized  as  implying  excellent  reliability. 

After  the  data  were  collected,  we  used  regression  methods  to  construct  sickness-at- 
admission  scales  and  to  adjust  outcomes  for  sickness  at  admission,  and  Likert  scales  and 
clinical  judgment  to  construct  explicit  scales  measuring  the  appropriateness  of  the  processes 
of  care  and  the  level  of  instability  at  discharge.  Some  items  that  went  into  these  scales  were 
relevant  to  all  patients,  some  only  to  subsets  of  patients  depending  upon  their  clinical  needs. 
We  concentrated  on  aspects  of  care  for  which  the  generally  recognized  standards  of  good  clin- 
ical practice  were  both  unambiguous  and  stable  over  time  from  1981  to  1986. 

We  also  developed  a  structured  implicit  review  form;  PRO  clinicians  used  this  instru- 
ment to  give  their  expert  judgments  about  the  quality  of  care  received  by  a  10  percent  ran- 
dom subsample  of  the  patients  whose  care  we  evaluated  using  explicit  criteria.  We  used 
psychometric  scaling  and  clinical  judgment  to  build  implicit  process  scales  and  regression  to 
examine  the  relation  between  explicit  and  implicit  process. 

In  every  phase  of  the  study  we  used  a  multidisciplinary  approach.  We  drew  on  clinical 
expertise  in  general  internal  medicine,  geriatrics,  cardiology,  pulmonary  medicine,  infectious 
diseases,  neurology,  orthopedics,  and  psychiatry  from  physicians  in  each  of  our  five  study 
states,  including  physicians  from  both  urban  and  rural  hospitals.  This  created  a  clinical  per- 
spective in  the  study  that  incorporated  both  academic  and  community-based  views  of  medical 
care.  This  clinical  perspective  was  complemented  by  expertise  in  statistics,  psychometrics, 
economics,  health  policy,  and  evaluation  sciences. 

The  remainder  of  this  report  gives  additional  details  on  our  design,  sampling,  and  field- 
work;  the  measurement  of  sickness  at  admission;  the  explicit  measurement  of  process  of  care; 
the  implicit  measurement  of  process  of  care;  patient  status  at  discharge;  patient  outcomes 
before  and  after  the  introduction  of  PPS;  and  a  summary  of  the  effects  of  prospective  pay- 
ment on  quality  of  care. 

REVIEW  OF  THE  LITERATURE  ON  PPS  EFFECTS 

When  we  began  our  study  in  1985,  little  was  known  about  the  effects  of  PPS  on  quality 
of  care.  Since  then,  however,  other  investigators  have  obtained  results  that  serve  as  context 
for  this  report.  Here,  we  review  the  most  recent  contributions  to  this  literature. 

Using  a  sample  of  646  U.S.  nonfederal,  short-term  general  hospitals  from  1980  to  1985, 
DesHarnais  et  al.  (1988)  found  that  Medicare  discharges  and  lengths  of  stay  declined 
significantly  after  the  introduction  of  PPS,  whereas  use  of  skilled  nursing  facilities  and  home 
health  care  after  discharge  increased  post-PPS.  Mayer-Oakes  et  al.  (1988)  studied  patients 
in  the  intensive  care  units  of  three  hospitals  before  and  after  PPS.  They  found  a  31  percent 
decrease  in  the  number  of  intensive  care  unit  beds  but  no  changes  in  patient  illness  or  treat- 
ment intensity  after  PPS  was  implemented.  They  found  a  reduction  in  lengths  of  stay,  both 
for  the  overall  hospitalization  and  for  the  use  of  the  intensive  care  unit.  Despite  the  reduc- 
tions in  lengths  of  stay,  they  found  no  significant  changes  in  mortality  inhospital  or  at  six 
months  after  hospitalization.  This  study  also  confirmed  the  work  of  DesHarnais  et  al.  in 
identifying  a  higher  percentage  of  patients  who  were  institutionalized  after  PPS.  More 
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recently,  Sager  et  al.  (1987,  1989)  used  age-specific  national  mortality  data  from  1981 
through  1985  to  report  changes  in  the  location  of  death  after  PPS  from  the  acute  care  hospi- 
tal setting  to  nursing  homes.  Guterman  et  al.  (1988)  found  a  reduction  in  the  number  of 
Medicare  short-stay  hospital  admissions,  a  reduction  in  lengths  of  stay,  and  an  increase  in 
number  of  discharges  to  skilled  nursing  facilities  using  HCFA's  Medicare  Statistical  System 
as  a  data  source. 

Fitzgerald  et  al.  (1987,  1988)  found  the  most  striking  differences  in  outcomes  after  PPS, 
in  their  analyses  of  discharge  disposition  and  prolonged  nursing  home  stay  for  hip  fracture 
patients  in  two  large  hospitals  before  and  after  PPS.  They  reported  (1988)  that  a  reduction 
in  the  number  of  physical  therapy  sessions  given  to  patients  after  PPS  accompanied  the 
reduction  in  lengths  of  stay.  Most  important,  they  found  an  increase  from  38  percent  to  60 
percent  in  the  proportion  of  patients  discharged  to  a  nursing  home,  and  an  increase  from  9 
percent  to  33  percent  in  the  proportion  of  patients  with  continued  nursing  home  care  at  one 
year  after  discharge.  These  results,  taken  from  events  at  a  single  hospital,  are  in  contrast  to 
those  of  Palmer  et  al.  (1989),  who  found  no  change  in  either  the  proportion  of  hip  fracture 
patients  discharged  to  a  nursing  home  or  the  proportion  in  a  nursing  home  at  six  months. 
However,  the  Palmer  et  al.  study  also  used  data  from  a  single  hospital,  albeit  one  with  quite 
different  characteristics  than  the  Fitzgerald  et  al.  hospital.  Gerety  et  al.  (1989)  studied  hip 
fracture  patients  in  a  tertiary  care  setting  and  found  shorter  lengths  of  stay  and  decreased 
functional  status  at  discharge  post-PPS.  Their  assessment  of  outcomes  at  one  year  after  hos- 
pitalization showed  no  difference  pre-  and  post-PPS  in  the  fraction  of  institutionalized 
patients.  These  results  emphasize  the  need  for  generalizable  evidence  on  the  effects  of  PPS 
on  quality  of  care.  We  have  tried  to  provide  such  evidence  in  this  report. 

GENERALIZABILITY  OF  OUR  FINDINGS 

In  choosing  six  conditions  for  study,  we  developed  the  following  criteria  characterizing 
desirable  conditions:  high  prevalence  in  the  Medicare  population;  high  mortality,  so  that  pat- 
terns of  care  leading  to  preventable  death  might  be  evident;  well-defined  diagnostic  criteria 
that  are  readily  available  from  data  in  the  medical  record;  high  likelihood  of  a  strong 
process-outcome  link,  so  that  any  changes  in  outcome  have  a  better  chance  of  being  correctly 
traced  back  to  changes  in  process;  and  relative  stability  of  consensus  on  what  constitutes 
good  care  across  the  entire  period  from  1981  to  1986  (Kahn  et  al.,  1988a). 

In  selecting  these  study  conditions,  we  recognized  that  it  would  be  ideal  to  study  condi- 
tions with  stable  diagnostic  and  therapeutic  modalities  across  the  study  periods.  We 
attempted  to  select  such  conditions  and  avoided  including  for  study  several  interesting  condi- 
tions because  of  apparent  changes  in  medical  practice  over  time.  However,  it  is  impossible  to 
state  for  a  certain  disease  that  no  changes  in  diagnoses  or  therapies  have  ensued  during  the 
period  from  1981  to  1986,  or  during  any  five-year  period.  The  diseases  we  selected,  however, 
were  selected,  among  other  reasons,  for  relative  stability  in  process  over  time.  Even  for  acute 
myocardial  infarction,  the  major  new  interventional  strategies  were  not  widely  diffused  by 
June  1986.  This  is  seen  in  Table  1.1  where  we  have  displayed  use  rates  for  four  "new"  inter- 
ventions. 

Of  these  interventions,  the  only  one  with  proven  efficacy  for  reducing  mortality  in  the 
patient  with  acute  myocardial  infarction  is  streptokinase.  Although  the  GISSI  trial  (1987) 
showed  a  reduction  in  mortality  at  one  year  for  patients  overall  with  acute  myocardial  infarc- 
tions (17.2  percent  for  patients  treated  with  streptokinase  and  19.0  percent  for  control 
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Table  1.1 

USE  OF  NEWER  TECHNOLOGIES  FOR  PATIENTS  WITH  ACUTE 
MYOCARDIAL  INFARCTION  PRE-  AND  POST-PPS 


Pre-  and 

Post-PPS 

Pre-PPS 

Post-PPS 

Combined 

Technology 

(n=l,416) 

(n=l,437) 

(n=2,853) 

Coronary  artery  bypass  surgery 

0.8 

2.7 

1.8 

Coronary  angiography 

2.7 

7.8 

5.3 

Angioplasty 

0.1 

2.2 

1.2 

Streptokinase  (includes  both  intravenous 

2.3 

5.5 

3.8 

and  intracoronary  use) 

patients,  p  =  0.008),  there  was  no  significant  difference  in  one-year  mortality  for  patients  65 
years  old  or  older.  It  would  be  difficult  to  attribute  changes  in  mortality  for  patients  with 
myocardial  infarction  post-PPS  compared  with  pre-PPS  to  the  use  of  streptokinase  both 
because  of  the  low  frequency  of  its  use  in  both  time  periods  (though  its  use  increased  post- 
PPS)  and  because  the  patients  in  our  study  were  all  65  years  old  or  older. 

To  be  conservative,  we  included  process  criteria  in  the  study  only  if  there  was  general 
agreement  among  our  physician  expert  consultants  that  a  particular  process  measure  was 
equally  relevant  in  1986  as  in  1981.  In  the  event  that  an  important  new  technology  was 
introduced  and  we  thought  it  was  important  to  include  the  use  of  that  technology  in  our  pro- 
cess measures,  the  process  criteria  were  developed  in  a  way  that  would  avoid  a  time-period 
bias.  For  example,  in  developing  the  process  criteria  for  cerebrovascular  accident,  we  recog- 
nized that  in  1986  many  institutions,  especially  academic  institutions,  believed  it  was  stan- 
dard practice  to  image  the  brain  of  very  sick  stroke  patients  with  a  CAT  scan.  However,  this 
clearly  was  not  standard  practice  in  1981.  In  the  early  1980s,  however,  it  was  standard  prac- 
tice to  image  the  brain  of  very  sick  stroke  patients  with  a  brain  scan  or  electroencephalogram 
(EEG),  or  to  perform  a  lumbar  puncture  or  carotid  angiography.  Thus,  our  process  criteria 
state:  "If  the  patient  with  cerebrovascular  accident  shows  posturing,  no  response  to  pain,  or 
coma,  then  do  any  of  the  following:  CAT  scan  of  the  brain,  carotid  angiography,  brain  scan, 
electroencephalogram,  or  a  lumbar  puncture."  This  way,  credit  is  assigned  in  the  scoring  of 
the  process  criteria  for  use  of  any  of  the  modalities  that  were  considered  standard  diagnostic 
measures  for  very  sick  stroke  patients  during  the  entire  period  from  1981-1986. 

A  second  example  of  how  we  used  a  new  technology  to  study  process,  but  avoided  a 
time-period  bias,  is  with  our  study  of  the  use  of  streptokinase  for  patients  with  acute  myocar- 
dial infarction.  We  do  not  assign  or  penalize  for  use  of  streptokinase  in  either  the  pre-  or  the 
post-PPS  periods.  However,  we  do  use  the  following  process  criteria:  "If  the  patient  is  given 
streptokinase,  then  documentation  should  be  made  in  the  medical  record  of  the  presence  or 
absence  of  a  bleeding  disorder."  (Streptokinase  is  contraindicated  in  patients  with  bleeding 
disorders.)  In  this  example,  the  use  of  streptokinase  does  not  contribute  to  a  higher  or  lower 
process  score.  However,  the  process  criteria  state  that  if  streptokinase  is  used,  it  should  be 
used  properly  (only  after  documentation  of  no  bleeding  disorder)  regardless  of  use  in  the  pre- 
or  post-PPS  period. 

Another  process  criteria  states,  "If  the  patient  is  given  streptokinase,  then  the  strepto- 
kinase should  be  started  within  six  hours  of  the  time  symptoms  were  first  noted."  Again, 
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process  credit  or  debit  is  not  assigned  for  use  of  streptokinase  but  for  proper  use  in  the  event 
that  the  drug  is  used. 

The  other  principle  that  guided  our  selection  of  diseases  was  the  desire  to  stratify  on 
how  the  disease  was  primarily  treated,  across  the  categories  medical,  surgical,  and  psychiat- 
ric. In  this  way  we  were  led  to  four  medical  conditions  (congestive  heart  failure,  acute  myo- 
cardial infarction,  pneumonia,  and  cerebrovascular  accident),  one  surgical  condition  (hip 
fracture),  and  one  psychiatric  condition  (depression).  In  the  sections  that  follow  we  present 
both  disease-specific  findings  and  results  obtained  by  aggregating  across  our  five  nonpsychi- 
atric  diseases. 

By  virtue  of  the  representativeness  of  our  patient  samples,  our  results  generalize 
directly  to  the  population  of  all  Medicare  patients  with  one  of  these  five  nonpsychiatric 
diseases — a  population  that  included  18  percent  of  all  Medicare  admissions  and  32  percent  of 
all  deaths  within  30  days  of  admission  in  fiscal  1986  (HCFA,  1986a).  But  do  our  results  gen- 
eralize outward  to  the  population  of  Medicare  patients  with  other  diseases,  and  thus  to  the 
entire  Medicare  cohort? 

Three  general  remarks  can  be  made  in  answering  this  question.  First,  in  searching  for 
process-outcome  links,  we  have  focused  mainly  on  connections  between  process  and  mortal- 
ity. We  cannot  comment  on  changes  in  quality  of  care  for  patients  with  terminal  conditions 
such  as  end-stage  cancer,  where  no  change  in  process  can  lead  to  a  better  mortality  outcome 
given  the  present  state  of  medical  knowledge.  For  such  patients,  quality  of  care  would  be 
defined  by  such  factors  as  functional  status  and  the  quality  of  the  dying  experience,  and  we 
did  not  measure  these  aspects  of  care.  Second,  our  results  probably  generalize  more  readily 
to  other  medical  conditions  than  to  other  surgical  and  psychiatric  diseases.  However,  for 
categories  of  process  that  show  consistent  patterns  of  change  across  all  of  our  conditions,  it  is 
hard  to  imagine  how  such  change  would  be  concentrated  only  in  those  conditions,  when  the 
same  components  of  care  (e.g.,  thoroughness  of  nursing  services,  appropriateness  of  ordering 
of  laboratory  tests,  and  completeness  of  physician  history  and  physical  examinations)  are  an 
integral  part  of  care  for  other  conditions  as  well. 

Third,  our  selection  of  diseases  makes  possible  a  kind  of  upper  bounding  or  a  fortiori 
argument  of  the  following  form:  By  concentrating  on  conditions  with  high  mortality  and 
likely  strong  process-outcome  links,  we  chose  diseases  for  which  it  seemed  that,  if  quality  of 
care  has  declined  under  PPS,  this  decline  would  likely  manifest  itself  in  process  and  outcome 
changes  in  patients  with  these  diseases.  Therefore,  if  we  do  not  see  a  large  decline,  the 
overall  detrimental  effects  of  PPS  aggregated  across  all  Medicare  patients  cannot  be  very 
large.  For  aspects  of  care  in  which  we  do  see  a  decline,  this  argument  is  of  no  help  in  pre- 
cisely estimating  its  overall  magnitude  for  all  Medicare  patients,  but  where  we  do  not  see  a 
decline,  we  will  have  learned  something  about  the  likely  effects  of  PPS  on  the  entire  Medi- 
care cohort. 

CAVEATS  ON  THE  INTERPRETATION  OF  OUR  RESULTS 

In  addition  to  the  above  remarks  about  generalizing  outward  from  the  diseases  we 
selected,  three  other  major  caveats  are  worth  bearing  in  mind. 

Our  design  involves  a  direct  comparison  between  the  quality  of  care  given  in  1981-1982 
and  in  1985-1986.  We  are  thus  able  to  fully  demonstrate  that  any  changes  in  quality  we  find 
are  associated  with  the  introduction  of  PPS,  but  definitively  identifying  which  of  those 
changes  were  caused  by  PPS  is  harder,  since  other  aspects  of  medical  care  besides  PPS  may 
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also  have  changed  from  1981  to  1986  (Vladeck,  1988).  We  have  attempted  to  deal  with  this 
by  measuring  quality  of  care  at  multiple  time  points  in  both  the  pre-  and  post-PPS  periods,  so 
that  secular  trends  in  medicine  that  were  in  place  before  PPS  was  implemented  may  be  at 
least  crudely  estimated  and  removed.  Our  approach  in  this  report  is  to  first  present  straight- 
forward comparisons  of  the  pre-  and  post-PPS  periods  and  to  sort  out  issues  of  trend  and 
causality  in  the  last  section. 

We  studied  quality  of  care  only  for  hospitalized  patients,  and  we  did  not  examine  ques- 
tions of  access  to  that  care.  We  cannot  address  whether  PPS  has  made  it  harder  or  easier  for 
certain  types  of  patients  to  obtain  high-quality  care,  an  important  part  of  the  total  PPS  policy 
picture. 

Finally,  our  post-PPS  data  are  from  1985  and  1986.  Although  all  hospitals  had  changed 
to  a  prospective  payment  system  by  1985,  payments  were  still  determined  in  part  by  the 
hospital's  own  costs  through  1986.  During  1985  and  1986  most  hospitals  were  still  profiting 
from  PPS.  Since  that  time  PPS  payments  have  been  tightened,  and  yearly  Medicare  pay- 
ment increases  to  hospitals  are  no  longer  keeping  up  with  the  inflation  rate  (Iglehard,  1986; 
ProPac,  1988).  On  the  other  hand,  changes  in  lengths  of  stay  associated  with  the  introduc- 
tion of  PPS  appear  in  recent  years  to  have  stabilized  and  have  not  continued  to  decline 
(Witsberger  and  Kominsky,  1990).  Our  results  may  be  suggestive  of  the  state  of  affairs  in 
1989,  but  it  seems  wise  to  continue  to  collect  clinically  detailed  data  to  monitor  sickness  at 
admission,  processes,  and  outcomes  on  a  regular  basis  as  long  as  prospective  payment  is  in 
place. 

DISCUSSION 

At  the  beginning  of  our  study,  we  faced  a  key  design  choice  that  has  implications  for 
future  data  collection  in  quality  of  care  monitoring:  Should  quality  of  care  studies  be  based 
on  detailed  clinical  review  of  medical  records,  or  on  administrative  data  gathered  routinely, 
or  on  both?  In  these  final  paragraphs,  we  lay  out  the  pros  and  cons  of  each  approach  and  dis- 
cuss the  reasoning  behind  the  choice  we  made  in  our  study. 

A  good  example  of  a  large  administrative  database  is  HCFA's  MedPAR  file  (HCFA, 
1986b),  which  contains  extensive  billing  information,  but  only  a  minor  amount  of  clinical 
detail,  on  every  Medicare  hospitalization.  Since  1987  HCFA  has  linked  this  administrative 
dataset  to  Social  Security  death  files  and  reported  hospital-level  mortality  rates  (HCFA, 
1987).  The  cost  per  case  of  clinically  detailed  review  of  medical  records  is  much  higher  than 
the  corresponding  cost  of  obtaining  MedPAR  information  linked  with  mortality  outcomes,  so 
that,  for  the  same  amount  of  money  that  it  would  take  to  examine  MedPAR  data  on  the 
entire  Medicare  population,  one  could  afford  to  subject  only  a  sample  from  this  population  to 
clinical  review. 

Constraints  imposed  by  the  cost  of  detailed  medical  record  review  may  limit  in  three 
ways  the  numbers  and  kinds  of  patients  studied  clinically.  As  we  detail  in  Sec.  II,  the  only 
cost-feasible  way  to  select  a  nationally  representative  sample  of  hospitalized  patients  for 
extensive  medical  record  review  is  through  multistage  cluster  sampling,  for  instance,  by 
choosing  samples  of  states,  geographic  areas  within  states,  hospitals  within  areas,  and 
patients  within  hospitals.  In  addition,  to  make  statements  about  quality  of  care  over  a 
multiple-year  period,  it  may  be  necessary  for  cost  reasons  to  further  restrict  sampling  with 
the  medical-record-based  approach  to  a  subset  of  the  years  of  interest.  Finally,  we  have  pre- 
viously argued  that  evaluating  the  appropriateness  of  medical  care  is  better  done  disease  by 
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disease,  so  that  with  a  limited  budget  one  must  also  choose  a  sample  of  diseases  on  which  to 
concentrate  quality  of  care  monitoring  (Kahn  et  al,  1988a). 

The  central  drawback  to  the  medical-record-based  approach  is  thus  that  its  findings  will 
necessarily  be  based  upon  a  sample  of  patients,  rather  than  upon  all  patients  of  interest  to 
the  study.  In  addition,  the  operational  complexities  of  using  a  medical-record-based 
approach  cause  a  longer  delay  in  making  the  data  available  for  use  than  is  seen  when  using 
administrative  data.  The  central  advantage  to  this  approach  is  the  rich  clinical  detail  one 
can  collect  on  sickness  at  admission,  appropriateness  of  the  processes  of  care  delivered,  insta- 
bility at  discharge,  and  outcomes.  Table  1.2  summarizes  the  advantages  and  disadvantages 
of  the  two  approaches.  Other  researchers  (Roos  et  al.,  1987)  have  come  to  similar  views.  In 
our  study  we  chose  the  medical-record-based  approach. 

We  encourage  readers  of  this  report  both  to  examine  our  findings  critically  and  to  reflect 
on  where  to  go  from  here  in  data  collection  and  analysis  for  quality  of  care  assessment  and 
assurance  among  Medicare  and  other  patients.  What  combination  of  clinical  detail  and 
administrative  data  would  lead  to  the  most  cost-effective  approach  to  monitoring  quality  of 
care  in  U.S.  hospitals  in  the  1990s,  as  the  effects  of  PPS  and  other  cost-containment  mea- 
sures become  stronger?  Should  HCFA  require  hospitals  to  report  clinical  information 
routinely,  along  with  billing  data?  If  so,  which  clinical  data?  If  not,  how  well  can  high-  and 
low-quality  facilities  be  identified  on  the  basis  only  of  administrative  information  linked  with 
mortality  files?  Answers  to  these  important  questions  are  not  yet  known. 


Table  1.2 

ADVANTAGES  AND  DISADVANTAGES  OF  QUALITY  OF  CARE  STUDIES 
BASED  ON  DETAILED  MEDICAL  RECORD  REVIEW  COMPARED 
WITH  THOSE  USING  ADMINISTRATIVE  DATA 


Clinically  Detailed 
Medical-Record-Based  Studies 


Studies  Based  on 
Administrative  Data 


Which  Is  Better? 


Limited  number  of  diseases 

Limited  number  of  states  and  hospitals 

Limited  time  period 

Moderate  sample  size  and  national 
representativeness 

Clinically  rich 

Sickness  at  admission 

Outcomes 

Appropriateness  of  processes  of  care 

Instability  at  discharge 

Long  delay  in  making  results  available 
for  publication 


All  diseases 

All  states  and  hospitals 

All  times 

The  entire  national 
dataset 

Clinically  poor 

Limited 

Mortality  only 

Almost  nothing 

Nothing 

Shorter  delay  in  making  results 
available  for  publication 


Administrative 
Administrative 
Administrative 
Administrative 

Medical-record-based 
Medical-record-based 
Medical-record-based 
Medical-record-based 
Medical-record-based 
Administrative 


II.  DESIGN,  SAMPLING,  AND  FIELDWORK 


David  Draper,  Ph.D.,  Katherine  L.  Kahn,  M.D.,  Ellen  J.  Reinisch,  M.S., 
Marjorie  J.  Sherwood,  M.D.,  Maureen  F.  Carney,  M.S., 
Jacqueline  Kosecoff,  Ph.D.,  Emmett  B.  Keeler,  Ph.D., 

William  H.  Rogers,  Ph.D.,  Harry  Savitt,  Ph.D., 
Harris  Allen,  Ph.D.,  Kenneth  B.  Wells,  M.D.,  M.P.H., 
David  Reboussin,  M.S.,  Robert  H.  Brook,  M.D.,  Sc.D. 


METHODS 

Choice  of  Treatment,  Control  Groups,  and  Study  Years 

PPS  was  not  introduced  in  1983  as  a  controlled  experiment.  Instead,  before  October 
1983  hospitals  were  reimbursed  for  treating  Medicare  patients  under  the  old  retrospective 
payment  system,  and  from  October  1983  to  September  1984  nearly  all  acute  care  general  hos- 
pitals were  phased  into  prospective  reimbursement.  The  exceptions  were  hospitals  in  the  so- 
called  waiver  states — Maryland,  Massachusetts,  New  Jersey,  and  New  York — where  reim- 
bursement alternatives  to  the  PPS  system  were  used  until  1986. 

Because  PPS  was  introduced  in  this  nonexperimental  way,  a  prospective  controlled  trial 
evaluating  its  effects  on  quality  of  care  was  not  possible,  and  we  instead  designed  a  retro- 
spective observational  study.  A  comparison  of  the  pre-  and  post-PPS  periods,  in  which  data 
on  Medicare  patients  before  and  after  1983  are  contrasted,  is  central  to  any  such  study. 
Designs  including  only  data-gathering  of  this  type  are  straightforward  but  have  a  potential 
confounding  factor:  The  effects  of  PPS  will  be  confounded  with  any  secular  trends  over  time 
in  the  quality  of  care,  unless  such  secular  trends  may  be  estimated  and  removed.  There  are 
two  main  ways  to  do  this:  either  (a)  estimate  the  trends  based  only  on  data  from  Medicare 
patients,  or  (b)  obtain  information  from  both  periods  in  parallel  on  a  control  group  of  patients 
unaffected  by  PPS  and  use  the  trends  evident  in  the  control  group  as  a  proxy  for  what  the 
Medicare  trends  would  have  been  in  the  absence  of  PPS. 

Both  of  these  options  have  strengths  and  weaknesses.  With  option  (a)  all  of  one's 
detailed  data-gathering  is  concentrated  in  the  Medicare  population,  which  provides  a  more 
accurate  look  at  what  actually  happened  to  Medicare  patients  pre-  and  post-PPS;  but  some 
form  of  extrapolation  of  pre-PPS  trends  is  needed  to  impute  what  would  have  been  observed 
post-PPS  if  PPS  had  not  been  implemented.  With  option  (b),  provided  the  control  patients 
are  truly  unaffected  by  PPS,  and  what  happens  to  them  is  a  good  proxy  for  what  would  have 
happened  to  Medicare  patients  in  the  absence  of  PPS,  no  extrapolation  is  needed,  because 
contemporaneous  control  information  is  gathered  directly  in  the  post-PPS  period;  but  a  group 
providing  such  a  proxy  may  be  difficult  to  find,  and  detailed  data-gathering  on  the  control 
group  diminishes  the  Medicare  sample  sizes. 

In  considering  option  (b)  we  examined  three  sources  of  non-PPS  data:  patients  treated 
in  the  waiver  states,  non-Medicare  patients  (e.g.,  those  55-64  years  old),  and  patients  from 
another  country  (e.g.,  Canada).  The  waiver  states  option  was  inadequate  because  the  reim- 
bursement programs  in  those  states  did  not  represent  an  absence  of  PPS  but  merely  one  or 
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another  variation  on  PPS.  The  non-Medicare  option  was  problematic,  because  changes  in 
hospital  behavior  induced  by  PPS  might  affect  these  patients  also.  Finally,  studying  changes 
in  quality  of  care  over  time  in  another  country  was  also  problematic,  because  the  health  care 
systems  in  other  countries  are  probably  too  different  to  use  trends  in  their  quality  of  care 
pre-  and  post-PPS  as  stand-ins  for  what  the  U.S.  experience  would  have  been  if  PPS  had  not 
been  implemented.  Nevertheless,  if  sufficient  funding  had  been  available,  we  would  have 
included  a  sample  of  the  Canadian  elderly  population  or  the  U.S.  55-64  year-old  cohort  as  a 
control  group. 

In  the  end,  however,  after  consultation  with  an  expert  panel,  we  chose  a  pre-post  design 
involving  only  Medicare  patients,  with  multiple  time  points  and  appeals  to  national  second- 
ary data  to  estimate  the  secular  trend. 

We  chose  as  our  pre-PPS  sampling  window  the  period  from  January  1,  1981,  to 
December  31,  1982,  and  as  post-PPS  window  the  period  from  July  1,  1985,  to  June  30,  1986. 
This  pre-PPS  window  was  the  furthest  back  in  time  we  could  go  and  still  be  assured  of  high 
availability  and  quality  of  medical  records  data,  and  our  chosen  post-PPS  window  was  the 
furthest  forward  we  could  go  given  the  lag  time  in  availability  of  postdischarge  data  crucial 
to  our  evaluation  of  quality.  Our  design  concentrated  50  percent  of  the  sampling  in  each  of 
the  pre-  and  post-PPS  periods,  as  follows:  20  percent  in  1981,  30  percent  in  1982,  and  25  per- 
cent in  each  of  1985  and  1986. 

Our  design  was  longitudinal  at  the  hospital  level  and  cross-sectional  at  the  patient 
level:  We  gathered  data  at  each  sampled  hospital  in  all  study  years,  with  different  patient 
cohorts  sampled  in  each  time  period  within  the  chosen  hospitals.  This  hospital-level  pairing 
increased  the  accuracy  of  the  pre-post  comparison  by  holding  the  hospital  factor  constant. 

We  have  elsewhere  advocated  the  measurement  of  quality  of  care  in  a  disease-specific 
way  (Kahn  et  al.,  1988a).  However,  it  is  expensive  to  develop  clinically  valid  disease-specific 
abstraction  forms  that  can  measure  outcomes,  sickness  at  admission,  and  appropriateness  of 
processes  of  care.  With  a  fixed  total  budget,  this  places  a  constraint  on  the  total  number  of 
diseases  sampled.  We  stratified  the  universe  of  diseases  to  which  Medicare  patients  are  sus- 
ceptible into  three  groups — medical,  surgical,  and  psychiatric — and  selected  six  diseases  for 
study:  four  medical,  one  surgical,  and  one  psychiatric.  The  conditions  we  selected  have  both 
high  frequency  and  high  death  rates  in  the  Medicare  population:  Together  our  six  diseases 
accounted  for  18  percent  of  all  Medicare  admissions  and  32  percent  of  all  Medicare  deaths 
within  30  days  of  admission  in  fiscal  1986.  (See  App.  A  for  further  discussion  of  disease 
selection  and  for  a  discussion  of  how  our  sampling  plan  changed  to  accommodate  the  study  of 
patients  with  depression.)  We  discuss  the  extent  to  which  our  findings  from  these  six 
diseases  generalize  outward  to  other  Medicare  patients  in  Sec.  I. 

Summary  of  Sampling  Plan 

We  used  a  stratified,  multistage  cluster  sampling  plan  with  four  levels  of  sampling 
hierarchy:  states,  areas  within  states,  hospitals  within  areas,  and  patients  within  hospitals. 
We  oversampled  hospitals  treating  many  Medicare  patients  and  chose  approximately  the 
same  number  of  patients  from  each  hospital  in  a  way  that  produced  patient-level  national 
representativeness.  In  our  sampling  design  we  chose  five  states,  4-8  areas  per  state  for  a 
total  of  30  areas  nationwide,  6-18  hospitals  per  area  for  a  total  of  approximately  60  hospitals 
per  state  and  300  hospitals  overall,  and  about  57  patients  per  hospital  for  a  total  of  approxi- 
mately 17,000  patients. 
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Choice  of  States.  We  selected  our  final  states  purposively,  subject  to  eligibility  cri- 
teria and  stratification  goals.  Eligibility  considerations  excluded  (a)  the  waiver  states  and  (b) 
states  with  either  too  few  hospitals  or  a  mixture  of  hospitals  that  was  either  too  urban  or  too 
rural.  The  main  stratification  goal  in  the  choice  of  states  was  geographic  diversity,  with  one 
state  from  each  region  of  the  country.  Our  final  sample  included  California,  Texas,  Indiana, 
Pennsylvania,  and  Florida  (see  App.  A). 

Choice  of  Areas.  We  had  to  choose  between  two  options  for  abstraction  of  data  from 
medical  records  in  this  study:  either  (1)  photocopying  the  chosen  records  and  sending  the 
copies  to  a  central  location  in  each  state  for  abstraction,  or  (2)  having  data  collectors  go  to  the 
chosen  hospitals  and  gather  the  data  in  person.  We  piloted  both  methods  and  discovered 
that  the  photocopy  quality  of  microfilmed  records,  which  made  up  a  nontrivial  portion  of  the 
pre-PPS  sample,  was  too  poor  to  permit  that  option.  Given  the  resulting  cost  restrictions  on 
data  collector  travel,  the  only  feasible  sampling  plan  involved  dividing  the  chosen  states  into 
geographic  areas  and  clustering  the  chosen  hospitals  in  a  sample  of  these  areas. 

We  used  geographic  diversity  within  state  and  six  hospital-level  factors  as  stratification 
variables  in  making  our  final  choice:  city  size,  percentages  of  Medicare  and  Medicaid 
patients,  hospital  size,  teaching  status,  and  hospital  ownership  (e.g.,  proprietary  compared 
with  nonprofit).  We  considered  a  large  number  of  designs,  each  with  a  total  of  20-30  areas, 
by  conducting  a  computer-aided  search  among  all  possible  choices  of  4-8  areas  in  each  state. 
Our  final  choice  (1)  had  substantial  geographic  and  urban-rural  diversity  within  each  state; 
(2)  permitted  oversampling  of  large  hospitals  and  facilities  with  a  high  percentage  of  poor 
patients  (as  indicated  by  Medicaid  status),  particularly  those  that  might  be  called  "classic 
city-county"  hospitals  (city  or  county  teaching  facilities  with  inner-city  locations  serving  a 
large  fraction  of  the  indigent),  so  that  we  might  learn  more  about  the  effects  of  PPS  on  poor 
patients;  and  (3)  permitted  the  selection  of  hospitals  that  were  nationally  representative  with 
respect  to  city  size,  percentage  of  Medicare  patients,  teaching  status,  and  ownership. 

Choice  of  Hospitals.  We  based  the  hospital-level  sampling  frame  on  the  1984  HCFA 
Provider  of  Services  file  (HCFA,  1986b).  We  used  three  eligibility  criteria  at  the  hospital 
level.  First,  we  restricted  sampling  to  short-term  acute  care  facilities,  thereby  excluding 
veterans',  military,  and  psychiatric  hospitals.  Second,  we  also  restricted  attention  to  hospi- 
tals that  were  in  existence  during  the  entire  period  from  1981  to  1986.  Third,  a  small 
number  of  hospitals  in  our  chosen  states  had  15  or  fewer  estimated  total  patients  per  year 
with  one  or  more  of  our  six  study  diseases  (based  on  HCFA's  1984  MedPAR  file)  and  were 
judged  to  be  too  small. 

The  final  hospitals  were  chosen  by  defining  a  4x4  stratification  grid  (Table  2.1)  indexed 
by  hospital  size,  city  size,  and  hospital  poverty  status.  We  defined  as  high-poverty  hospitals 
those  whose  percentage  of  Medicaid  patients  was  at  or  above  the  90th  percentile  of  the  per- 
centage of  Medicaid  patients  in  the  state  in  which  the  hospital  was  located.  The  goal  was 
representativeness  with  respect  to  hospital  size  and  city  size,  and  a  double  oversampling  of 
both  high-poverty  hospitals  and  classic  city-county  facilities.  Hospitals  were  then  chosen  by 
a  restricted  randomization  procedure  that  attempted  to  maximize  representativeness  with 
respect  to  ownership,  teaching  intensity,  and  percentage  of  Medicare  patients,  while  varying 
the  number  of  hospitals  per  area  from  6  to  18  in  such  a  way  that  the  stratification  targets  in 
Table  2.1  were  achieved.  In  this  manner  300  hospitals  were  chosen. 

Almost  all  of  the  300  selected  hospitals  agreed  to  participate  in  the  study.  Five  hospi- 
tals refused  and  were  replaced  by  hospitals  in  the  same  area  with  similar  characteristics. 
Three  other  hospitals  also  refused  to  participate  and  were  not  replaced,  resulting  in  a  final 
sample  of  297  hospitals  and  a  hospital-level  participation  rate  of  97  percent  (297/305). 
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Table  2.1 

NATIONAL  COMPARED  WITH  SAMPLE  BREAKDOWN  IN  THE  4x4 
STRATIFICATION  GRID3 


Rural  Urban 


Hospital 

Not 

Not 

Size'' 

Poor 

Poor 

Poor 

Poor 

Total 

Very  small 

0.8 

5.0 

0.5 

4.1 

10 

O.D 

1  H 

9  9 

L.\J 

H.D 

n  a. 

O.D 

q 

V/.O 

7  K 

1  9 

Q  A. 

1  q 

1.7 

7.8 

5.8 

6.2 

21 

1.2 

7.7 

1.1 

9.3 

19 

Medium 

0.6 

5.8 

4.4 

41 

51 

0.0 

5.0 

7.3 

37 

49 

0.0 

6.1 

4.5 

41 

52 

Large 

0.0 

0.3 

1.8 

18 

20 

0.0 

0.0 

1.6 

18 

19 

0.0 

0.0 

1.8 

18 

20 

Total 

2.1 

19 

7.9 

73 

100 

2.6 

18 

16.5 

63 

100 

2.2 

18 

7.7 

72 

100 

The  top  figure  in  each  cell  in  this  table  is  the  national  prevalence 
of  patients  at  hospitals  of  the  indicated  type;  the  middle  figure  is  the 
raw,  unweighted  sample  prevalence;  and  the  bottom  is  the  reweighted 
sample  prevalence  using  weights  chosen  to  approximately  reproduce 
the  national  prevalences.  The  national  figures  were  arrived  at  by 
weighting  hospital-level  data  from  the  1984  HCFA  Provider  of  Ser- 
vices file  by  each  hospital's  number  of  certified  beds,  adjusted  for  its 
Medicare  volume  and  occupancy  rate.  Thus  the  top  number  in  the 
upper  left-hand  corner  of  the  table,  for  instance,  means  that  roughly 
0.8  percent  of  the  nation's  Medicare  patients  are  treated  in  very  small 
rural  poor  hospitals,  not  that  0.8  percent  of  the  nation's  hospitals  are 
very  small,  rural  and  poor.  The  cutoffs  for  hospital  size  are  as  follows: 
very  small  (adjusted  beds  fewer  than  22);  small  (between  23  and  49); 
medium  (between  50  and  154);  and  large  (more  than  154). 

dumber  of  beds  adjusted  for  Medicare  volume.  Adjusted  beds  = 
the  number  of  certified  beds,  multiplied  by  the  hospital's  percentage  of 
Medicare  patients. 


Choice  of  Patients.  We  based  the  patient-level  sampling  frame  on  lists,  generated  by 
the  chosen  hospitals,  of  patients  hospitalized  in  each  study  year  with  one  of  the  study 
diseases,  as  indicated  by  ICD-9-CM  codes  (Table  2.2).  Twelve  hospitals  were  unable  to  pro- 
vide such  lists,  so  we  built  the  patient  sampling  frames  for  those  hospitals  from  HCFA's 
MedPAR  file.  We  chose  simple  random  samples  of  patients  in  the  cells  of  a  3x6  grid  indexed 
by  study  period  (1981,  1982,  1985-1986)  and  disease.  The  goal  was  balance  across  all 
diseases,  with  nine  or  ten  patients  chosen  from  each  disease  category  in  each  hospital,  dis- 
tributed within  each  disease  across  study  periods  in  the  approximate  proportions  20  percent 
(1981),  30  percent  (1982),  and  50  percent  (1985-1986).  Our  final  sample  had  51-62  patients 
per  hospital  (except  for  12  small  hospitals,  where  the  number  of  patients  per  hospital  ranged 
from  17  to  50),  with  an  average  of  approximately  57  patients  per  hospital  and  a  total  of 
16,758  patients,  slightly  fewer  than  our  target  of  17,000. 
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Table  2.2 


ICD-9-CM  CODES  DEFINING  THE  SIX  STUDY  DISEASES 


Til  DQ  Q  on 

Codes 

Congestive  heart  failure 

428.xxa 

Acute  myocardial  infarction 

410.xx 

Pneumonia 

480.9x 

481.xx 

A  GO  w 

483.xx 

485.xx 

486.xx 

487.0x 

507.0x 

510.xx 

511. lx 

oio.ua 

7QQ  1  v 

Cerebrovascular  accident 

431.xx 

434.xx 

436.xx 

Hip  fracture 

820.0xb 

820.2x 

820.8x 

Depression 

296.2x 

296.3x 

298.0x 

300.4x 

309.0x 

309.1x 

311.xx 

lx  =  any  value  0  to  9  or  no  value. 
'Except  820.01. 


To  meet  our  goal  of  57  patients  per  hospital  we  had  to  examine  a  somewhat  larger 
number  of  records,  because  some  records  were  unavailable  for  study  and  others  did  not  meet 
our  clinical  inclusion  criteria  (see  below).  In  total,  22,795  records  were  requested  from  the 
participating  hospitals. 

Hospitals  were  able  to  find  the  requested  medical  records  in  almost  all  cases.  Of  the 
22,795  medical  records  requested,  870  (3.8  percent  of  the  requested  records)  were  not  avail- 
able for  review,  leaving  21,925  total  records  reviewed.  Table  2.3  summarizes  the  reasons 
given  for  unavailability.  As  expected,  the  number  of  unavailable  records  was  higher  in  1981 
(6.6  percent  of  those  requested)  and  1982  (5.0  percent)  than  in  1985-1986  (1.7  percent).  The 
number  of  unavailable  records  was  sufficiently  small  that  any  resulting  bias  would  be  too 
small  to  significantly  change  the  study's  major  findings. 

Data  Collector  Training,  Abstraction,  and  Monitoring 

Nurses  and  medical  records  personnel  experienced  with  clinical  data  were  selected  by 
the  PROs  in  the  chosen  states  to  become  data  collectors.  After  demonstrating  adequate 
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Table  2.3 

REASONS  WHY  SOME  MEDICAL  RECORDS  WERE  UNAVAILABLE,  BY  YEAR 


Reason 

1981 

1982 

1985-1986 

Total 

A.  Couldn't  find  patient  with  that  admission/disease/timeframe. 

Patient's  record  was  pulled,  but  the  requested  admission  could  not  be 
found;  or  patient's  record  was  not  pulled  because  the  requested  admission 
was  not  found  on  the  indexes;  or  mistaken  medical  record  number  was  recorded 
and  record  pulled  was  for  the  wrong  patient,  e.g.,  one  who  was  under  65 
years  of  age  or  had  the  wrong  diagnosis;  or  patient's  record  was  located  on 
microfilm/microfiche,  but  the  requested  admission  was  not  in  the  record;  or 
the  medical  record  number  was  not  a  valid  number  for  the  time  frame 
requested  or  no  patient  has  the  number;  or  requested  medical  record  number 
was  outpatient  only. 

78 

59 

43 

180 

B.  Record  sent  out  to  be  microfiched. 

Record  not  available  because  it  was  being  microfiched. 

21 

20 

5 

46 

C.  Record  cannot  be  found. 

Medical  records  department  was  unable  to  locate  the  chart  (used  this 
code  when  more  complete  description  of  the  location  problem  was  not  given  or 
was  not  understandable);  or  indexes  indicated  that  record  was  stored  off  site 
but  the  record  could  not  be  found  or  could  not  be  retrieved;  or  patient 
died  in  the  hospital  and  the  chart,  which  should  have  been  archived  according 
to  ho^nitfil  nolirv  could  not  hp  located*  or  record  was  nullpd  for  RAND  ^tudv 
but  no  one  could  find  it  when  review  coordinator  arrived  to  do  the  abstractions. 

162 

184 

97 

443 

D.  Only  the  emergency  room  report  could  be  found. 

Patient  died  shortly  after  admission.  Only  the  emergency  room  report  could 

be  found. 

0 

0 

1 

1 

E.  Problem  with  numbering  system  on  records  or  on  microfiche. 

Hospital  changed  system  of  numbering  records  and  could  not  locate  some 
1981  and  1982  records  without  the  patients'  names;  or  unable  to  locate  because 
of  problem  with  numbering  on  microfiche  jacket. 

30 

33 

0 

63 

F.  Record  was  in  use/out  for  other  review. 

Board  of  Health  Review  Team  has  the  record;  or  patient  in  the  hospital 
and/or  the  chart  was  in  use  when  the  review  coordinator  needed  it;  or  record 
was  impounded  by  court  during  litigation  and  never  returned. 

2 

3 

6 

11 

G.  Record  found  too  late. 

Record  found  after  form  was  filled  out  or  after  review  coordinator  completed 
quota  for  the  disease. 

15 

24 

15 

54 

H.  Parts  of  record  missing  or  unreadable. 

Microfilm  roll  was  located  but  would  not  feed  into  the  reader/printer;  or 
chart  was  pulled  but  parts  needed  for  the  study  were  missing. 

2 

9 

1 

12 

I.  Record  shredded  in  error. 

Chart  was  shredded  by  mistake. 

3 

0 

0 

3 

J.  MedPAR  information  not  sufficient  to  locate  the  record. 
MedPAR  information  was  not  sufficient  to  locate  the  chart. 

2 

3 

0 

5 

K.  Shredded  after  5  years  by  policy/confiscated  in  takeover. 

Chart  was  destroyed  per  hospital  policy  of  shredding  after  5  years;  or  record 
was  confiscated  in  hospital  takeover. 

2 

5 

1 

8 

L.  Patient  and  admission  same  as  one  already  pulled. 

Patient  and  admission  were  the  same  as  one  already  pulled  for  this 
hospital/disease. 

2 

6 

2 

10 

M.  No  documentation  received  on  why  record  was  unavailable. 

12 

7 

15 

34 

Total  requested  records 

5,012 

6,999 

10,784 

22,795 

Total  unavailable  records 

331 

353 

186 

870 

Percent  unavailable  records 

6.6 

5.0 

1.7 

3.8 

Total  reviewed  records 

4,681 

6,646 

10,598 

21,925 
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skills,  52  data  collectors  participated  in  training  that  lasted  17  days  and  covered  all  six 
diseases,  with  at  least  two  days  focused  exclusively  on  the  study  of  each  disease. 

During  the  period  of  data  collection,  we  monitored  abstractors  with  a  series  of  interrater 
reliability  studies1  and  gave  feedback  to  data  collectors  who  were  identified  as  having  prob- 
lems. A  project  manager  who  was  familiar  with  the  hospital  in  which  the  abstraction  took 
place  supervised  the  work.  In  addition,  each  completed  abstraction  form  was  reviewed  by 
both  a  physician  and  a  nurse  for  an  average  of  30  minutes  per  record,  to  assess  internal  con- 
sistency and  to  assure  that  coding  was  consistent  with  supporting  clinical  data.  Certain 
types  of  clinical  information  from  the  medical  record  (e.g.,  the  exact  words  characterizing  a 
patient's  stroke  if  one  occurred)  were  written  verbatim  into  the  abstraction  form,  and 
specified  pieces  of  data  (e.g.,  chest  X-ray  reports,  admission  histories  and  physicals,  and 
discharge  summaries)  were  photocopied  and  attached  to  the  form.  Discrepancies  that  could 
not  be  resolved  during  the  review  process  were  returned  for  reabstraction.  Physicians  inter- 
preted photocopies  of  X-ray  reports  and  some  EKG  tracings  and  reports. 

The  research  team  visited  data  collectors  in  the  field  at  least  once  during  the  data  col- 
lection period.  A  supervisory  project  manager  spoke  with  the  research  team  in  each  state  at 
least  weekly.  In  addition,  we  invited  data  collectors  and  project  managers  to  call  a  RAND 
telephone  hotline  and  speak  with  a  physician  about  any  questions  that  might  arise  in  the 
field  as  they  used  the  abstraction  form.  Some  data  collectors  used  the  phone  calls  to  clarify 
the  meaning  of  an  item  in  the  abstraction  form  or  guidelines.  Others  called  to  discuss  the 
best  means  for  abstracting  data  given  a  patient  situation  that  had  not  been  previously  antici- 
pated. 

To  maintain  confidentiality  of  hospitals,  patients,  and  physicians,  coded  identifiers  were 
assigned  to  hospitals  and  patients.  No  identifying  information  about  physicians  was 
obtained. 

Each  disease-specific  abstraction  form  (Kahn  et  al.,  1988b;  Kosecoff  et  al.,  1988;  Roth  et 
al.,  1988;  Rubenstein  et  al.,  1988;  Sherwood  et  al.,  1988;  and  Wells  et  al,  1988)  contained 
approximately  700  items  grouped  into  five  categories:  inclusion  criteria,  demographics  and 
sickness  at  admission,  explicit  information  about  the  processes  of  care  given  to  the  patient, 
patient  outcomes,  and  patient  status  at  discharge.  We  discuss  the  inclusion  criteria  below; 
more  details  about  data  elements  in  the  other  four  categories  can  be  found  in  other  sections 
of  this  report.  The  abstraction  forms  took  an  average  of  90  minutes  per  record  to  complete. 
(See  App.  A  for  further  discussion  of  instrument  development  and  data  collector  perfor- 
mance.) 


JBy  comparing  national  and  sample  patient-level  prevalences  in  cross-tabulations  indexed  by  hospital  size,  city 
size,  poverty  status,  teaching  intensity,  hospital  ownership,  and  percentages  of  Medicare  and  Medicaid  patients,  we 
created  weights  that  compensated  for  our  oversampling  of  high-poverty  and  classic  city-county  facilities  and  for 
small  unintentional  discrepancies  between  the  national  and  sample  Medicare  cohorts.  Table  2.1  presents 
unweighted  and  weighted  sample  prevalences  in  the  4x4  stratification  grid  used  to  choose  the  hospitals.  The 
weights  were  computed  by  forming  ratios  of  national  to  sample  counts  in  Table  2.1  and  modifying  the  resulting 
values  slightly  to  improve  representativeness,  after  reweighting,  on  the  other  variables  of  interest  (teaching  inten- 
sity, hospital  ownership,  and  percentage  of  Medicare  patients)  not  factored  into  Table  2.1. 

We  chose  a  sample  of  the  items  on  our  abstraction  forms  for  interrater  reliability  studies.  Table  2.6,  below,  sum- 
marizes our  findings  with  percentage  agreement  and  kappa  statistics  (AHA,  1984).  A  few  kappa  values  were  nega- 
tive, even  though  the  corresponding  percentage  agreement  values  were  high.  This  occurred  mainly  when  there  was 
so  little  variation  in  response  to  an  item  across  data  collectors  that  kappa  was  almost  undefined  (see  footnote  d  in 
Table  2.6).  Such  values  were  truncated  to  zero  in  the  table. 
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Inclusion  Criteria 

We  used  inclusion  criteria  to  assure  a  homogeneous  group  of  patients  with  the  chosen 
diseases  in  the  pre-  and  post-PPS  periods.  To  be  included  in  one  of  our  disease-specific  sam- 
ples, patients  had  to  be  at  least  65  years  of  age,  admitted  during  a  study  year,  and  hospital- 
ized for  the  indicated  disease.  Patients  with  the  study  disease  as  a  complication  of  hospital- 
ization, rather  than  as  a  reason  for  admission,  were  not  eligible  for  the  sample.  Except  for 
depression,  we  did  not  include  patients  who  were  transferred  from  another  acute  care  gen- 
eral hospital,  because  we  did  not  have  access  to  the  medical  record  from  the  initial  hospital 
stay.  We  also  excluded  patients  with  major  competing  conditions  (e.g.,  end-stage  cancer  in  a 
patient  with  congestive  heart  failure)  that  were  likely  to  determine  process  and  outcome  for 
the  patient  regardless  of  the  presence  of  the  study  disease. 

Of  the  21,925  reviewed  records,  24  percent  (5,167)  were  excluded  (Table  2.4).  The  frac- 
tion of  records  excluded  in  the  post-PPS  period  (21  percent)  was  significantly  lower  than  the 
fraction  excluded  in  the  pre-PPS  period  (26  percent).  Table  2.5  shows  the  most  frequent 
causes  of  exclusion — clinically  detailed  review  of  the  medical  record  showed  that  the  patient 
did  not  have  symptoms  or  signs  of  the  disease  suggested  by  the  hospitals'  assignment  of 
ICD-9-CM  codes.  For  example,  17  percent  of  the  patients  with  congestive  heart  failure  coded 
on  their  medical  records  did  not  have  symptoms  of  congestive  heart  failure  at  admission,  i.e., 
did  not  have  either  chest  X-ray  evidence  for  heart  failure  or  leg  edema  at  admission.  (See 
App.  A  for  further  details  about  disease-specific  inclusion  and  exclusion  criteria  and  for  a  dis- 
cussion of  acute  care  hospital  transfer  and  PPS  effect.) 

Secondary  Data  Collection 

In  addition  to  the  clinical  primary  data  described  above,  we  also  collected  secondary 
data  of  two  types:  (1)  information  about  postdischarge  outcomes  that  we  merged  with  the 
records  in  our  primary  database,  to  examine  the  effects  of  PPS  on  outcomes  during  the  year 
following  hospitalization;  and  (2)  national  data  on  mortality  trends  in  our  study  diseases  over 
the  period  from  1980  to  1986,  with  which  to  compare  our  sample  data  for  validation  purposes. 


Table  2.4 

PERCENTAGE  OF  RECORDS  EXCLUDED,  BY  DISEASE  AND  TIME  PERIOD 


Disease 

1981 

1982 

1985-1986 

Total 

Congestive  heart  failure 

29.8 

28.4 

19.7 

24.5 

Acute  myocardial  infarction 

18.7 

16.7 

14.8 

16.2 

Pneumonia 

39.3 

38.1 

35.9 

37.3 

Cardiovascular  accident 

22.4 

21.8 

13.1 

17.7 

Hip  fracture 

21.8 

18.3 

14.9 

17.4 

Depression 

21.7 

25.9 

24.2 

24.4 

Total  number  excluded 

1,266 

1,677 

2,244 

5,187 

(26.2%) 

(25.5%) 

(21.2%) 

(23.6%) 

Total  number  included 

3,455 

4,949 

8,354 

16,758 

Total  number  excluded  +  included 

4,721 

6,626 

10,598 

21,925 

^he  percentages  given  are  of  the  form  (number  of  excluded  records/number 
of  included  +  excluded  records). 
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Table  2.5 

FREQUENT  REASONS  FOR  RECORD  EXCLUSION,  BY  DISEASE 

(In  percent) 


Reason 

Congestive 

ncalX 

Failure 

Acute 
iviyocarciiai 
Infarction 

Pneumonia 

Cerebro- 

vascular 

Accident 

i  il  p 

Fracture 

±JcL)l  tja- 

sion 

Five 
Diseases 

XT     jl               •  i      1  ■         J    J         *  iJ 

Not  hospitalized  during  study  year 

0.3 

0.4 

0.2 

0.3 

0.3 

0.2 

A  O 
0.O 

No  eligible  ICD-9-CM  code 

i  a 

1.0 

1.0 

1.0 

1  o 

1.0 

Z.O 

1  C 

1.0 

Age  <  65 

A  o 

0.2 

0.3 

A  £ 
0.0 

0.O 

A  1 

0.1 

A  T 
U.  / 

A  A 
0.4 

Transferred  from  another  acute 
care  hospital 

1.8 

4.1 

1.8 

3.3 

2.2 

ND 

2.6 

^Jn  c\rm  T^^nm  c/ci  cm  c  nr  HiQonQO 
I'lU  oyilipiUIIlb/olgllb  Ul  Ulscasc  ttL 

admission 

16.8 

6.5 

27.5 

11.0 

2.2 

10.9 

12.5 

Acute  myocardial  infarction  at 
admission 

0.7 

ND 

0.2 

0.3 

0.2 

ND 

0.4 

Other  competing  conditionsa 

1.3 

2.2 

3.6 

0.6 

10.3 

6.4 

4.1 

Poor  prognosis  or  active  cancer 

1.8 

ND 

ND 

ND 

ND 

1.2 

1.5 

Total  number  excluded 

916 

550 

1,632 

608 

581 

880 

5,167 

Total  number  included 

2,824 

2,853 

2,749 

2,824 

2,762 

2,746 

16,758 

Total  number  excluded  +  included 

3,740 

3,403 

4,381 

3,432 

3,343 

3,626 

21,925 

NOTE:  ND  signifies  no  data.  The  disease-specific  abstraction  form  did  not  specify  this  as  an  exclusion  criterion 
and  therefore  no  data  were  collected  on  this  item. 

aOther  competing  conditions  included  important  clinical  problems  present  at  admission  that  were  likely  to 
influence  process  or  outcome  in  ways  not  seen  in  patients  without  these  conditions.  For  example,  the  presence  of 
bilateral  hip  fracture  or  multiple  trauma  were  exclusion  criteria  for  hip  fracture  patients. 


We  compiled  three  kinds  of  postdischarge  outcome  information:  mortality,  hospital  read- 
missions,  and  nursing  home  stays.  To  obtain  postdischarge  mortality  data,  we  used  patients' 
last  names,  first  names,  dates  of  birth,  and  Health  Insurance  Claim  (HIC)  numbers  from  the 
medical  record  to  match  our  sampled  patients  with  the  corresponding  records  in  HCFA's  Health 
Insurance  Master  (HIM)  file  (HCFA,  1986b).  We  were  able  to  obtain  accurately  merged  post- 
discharge  mortality  information  on  92  percent  of  our  sample.  We  give  additional  details  on  the 
merging  of  our  primary  data  with  postdischarge  databases  in  Sec.  VII. 

To  validate  our  sampling  with  respect  to  mortality,  we  obtained  data  from  HCFA's 
MedPAR  file  on  all  patients  hospitalized  with  the  five  selected  diseases  (n  =  2,062,610)  in  the 
27-quarter  period  from  January  1980  to  September  1986.  Variables  we  extracted  from  the 
MedPAR  file  included  death  status  within  30  days  of  admission;  gender;  age  (65-69;  70-74; 
75-79;  80-84;  85-89;  and  90+);  and  DRG.  We  established  a  correspondence  between  our 
ICD-9-CM-based  disease  definitions  (Table  2.2)  and  DRGs  and  used  these  data  to  calculate 
age/sex-specific,  and  age/sex-adjusted,  30-day  death  rates  for  each  disease,  and  in  the  aggre- 
gate across  diseases,  in  each  of  the  27  quarters. 

We  then  used  the  postdischarge  mortality  data  on  the  sampled  patients  to  compute 
age/sex-adjusted  quarterly  sample  mortality  rates  in  the  30-day  period  following  admission, 
by  disease  and  in  the  aggregate  across  diseases.  To  compare  these  data  with  the  national 
MedPAR  mortality  data  described  above,  we  plotted  the  national  age-  and  sex-adjusted  30- 
day  quarterly  mortality  series  on  the  same  graph  with  the  quarterly  30-day  mortality  data 
from  our  study  sample,  and  marked  off  error  bars  two  standard  errors  either  way  from  the 
observed  sample  mortality  series.  We  estimate  that  noncomparabilities  between  national 
and  sample  mortality  values  arising  from  our  use  of  inclusion  criteria  were  small. 
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Effects  of  Sampling  Plan  on  Analysis 

The  sampling  plan  we  employed  had  four  features  requiring  special  attention  during 
the  analysis.  We  now  describe  these  features  and  how  we  dealt  with  them.  (See  App.  A  for 
further  discussion  of  the  actual  hospital  and  patient  sample  plan  and  the  special  features  of 
the  sampling  plan.) 

1.  Our  oversampling  of  patients  from  hospitals  serving  an  unusually  large  fraction  of 
Medicaid  patients  would  yield  somewhat  biased  raw  findings  if  the  quality  of  care  in  high- 
Medicaid  facilities  differs  substantially  from  that  in  other  hospitals.  To  arrive  at  nationally 
representative  findings,  it  was  necessary  to  reweight  our  raw  patient-level  results,  giving 
less  weight  to  patients  from  high-Medicaid  facilities.  We  describe  above  how  we  calculated 
the  weights  for  this  purpose.  The  weighted  and  unweighted  results  differed  little  in  most  of 
our  major  analyses.  In  what  follows  in  this  and  subsequent  sections,  for  simplicity  we 
present  unweighted  findings  unless  otherwise  indicated. 

2.  The  clustering  of  our  sampled  patients  in  only  five  states,  30  areas,  and  297  hospitals 
has  implications  for  the  accuracy  of  our  results.  If  there  is  more  similarity  on  average 
between  two  patients  in  the  same  state,  area,  or  hospital  in  the  care  they  receive  than 
between  two  patients  in  different  states,  areas,  or  hospitals,  and  no  adjustment  for  this  so- 
called  intracluster  correlation  is  made  (Cochran,  1977),  the  result  will  be  an  overstatement  of 
the  precision  of  our  findings.  To  adjust  for  the  clustering  and  produce  standard  errors  that 
accurately  reflect  the  information  content  of  our  sample,  we  (a)  calculated  provisional  stan- 
dard errors  for  all  of  our  major  estimates  as  if  we  had  gathered  our  data  using  simple 
random  sampling;  (b)  computed  "inflation  factors"  based  on  intracluster  correlations  that 
measured  the  amount  of  information  in  our  sample  relative  to  that  obtained  in  a  simple  ran- 
dom sample  of  the  same  size;  and  (c)  multiplied  our  provisional  standard  errors  by  the 
inflation  factors,  thereby  adjusting  the  significance  of  our  results  downward.  The  inflation 
factors  ranged  from  1.1,  for  outcomes  such  as  inhospital  and  30-day  deaths,  to  2.9  for  our 
composite  score,  aggregating  across  diseases,  which  summarizes  the  quality  of  the  processes 
of  inhospital  care.  The  latter  inflation  factor,  which  derives  from  an  unusually  high  intra- 
cluster correlation  of  0.16,  indicates  a  remarkable  degree  of  homogeneity  within  hospitals  in 
the  processes  of  care  rendered. 

3.  The  longitudinal  nature  of  the  hospital  sampling,  in  which  we  gathered  data  in  all 
our  hospitals  in  both  the  pre-  and  post-PPS  periods,  also  had  implications  for  the  accuracy  of 
our  results.  Holding  the  hospital  factor  constant  acted  upon  the  precision  of  our  findings  in  a 
manner  opposite  to  that  of  the  clustering:  The  latter  decreased  accuracy,  when  compared 
with  independent  simple  random  sampling  in  each  of  the  pre-  and  post-PPS  periods,  and  the 
former  increased  accuracy,  because  of  positive  correlations  over  time  in  the  patterns  of  care 
given  to  patients  within  hospitals.  We  computed  "deflation  factors"  to  adjust  simple  random 
sampling  standard  errors  for  the  longitudinal  hospital  sampling  and  found  that  the  cluster- 
ing and  longitudinal  effects  approximately  canceled  each  other  in  our  analyses.  Thus  we 
were  able  to  analyze  the  data  essentially  as  if  they  had  been  gathered  with  simple  random 
sampling  in  a  cross-sectional  fashion. 

4.  In  this  report  we  present  results  both  at  the  disease-specific  level  and  in  the  aggre- 
gate across  diseases.  Two  issues  arise  in  producing  across-disease  summaries:  reweighting 
the  diseases  back  to  their  actual  frequencies  in  the  Medicare  population,  and  properly 
accounting  for  the  degree  to  which  pre-  and  post-PPS  differences  themselves  differ  by  disease 
(i.e.,  accounting  for  interactions  between  the  pre-  and  post-PPS  periods  and  disease  factors). 
Reweighting  by  disease  prevalence  is  potentially  necessary  because  we  took  samples  of 
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roughly  equal  size  in  each  of  the  six  diseases,  even  though  congestive  heart  failure  is  more 
than  three  times  more  frequent  among  Medicare  patients  than  hip  fracture,  for  example.  We 
computed  weights  necessary  to  reweight  our  raw  findings  back  to  the  population  of  all  Medi- 
care patients  with  one  of  the  six  diseases  and  did  sensitivity  analyses  to  see  how  much  this 
reweighting  affected  our  results.  In  most  cases  we  examined,  the  weighted  and  unweighted 
results  differed  little.  We  therefore  report  unweighted  findings  unless  otherwise  noted.  On 
pre-  and  post-PPS  interactions  by  disease  (a)  we  have  tried  not  to  emphasize  across-disease 
summaries  when  the  interactions  are  large,  i.e.,  when  there  is  serious  disagreement  among 
the  diseases  in  the  size  of  the  pre-  and  post-PPS  difference,  and  (b)  we  have  tried  to  conserva- 
tively report  the  significance  of  the  pre-  and  post-PPS  effects  when  the  pre-  and  post-PPS 
interactions  by  disease  are  small  to  moderate,  by  adjusting  the  pre-  and  post-PPS 
significance  in  a  manner  consistent  with  the  size  of  the  interaction  (Scheffe,  1959). 

RESULTS 

We  present  results  here  based  on  the  five  nonpsychiatric  diseases;  we  will  report  on 
depression  results  at  a  later  date.  The  final  sample  size  excluding  depression  patients  was 
14,012.  We  obtained  accurate  postdischarge  mortality  data  on  12,823  of  these  patients. 

Interrater  Reliabilities 

We  assessed  interrater  reliabilities  for  each  disease  using  both  records  known  to  the 
data  collectors  to  be  test  cases  ("known"  records)  and  records  on  which  they  did  not  know 
they  were  being  monitored  ("unknown"  records).  Table  2.6  summarizes  our  findings.  For  ten 
known  reliability  records  (the  "gold-standard"  columns  of  Table  2.6),  each  rated  by  an  aver- 
age of  47  different  data  collectors,  we  found  across-disease  item-level  kappa  scores  averaging 
0.86.  For  162  unknown  records  rated  by  two  data  collectors  (the  "field  data"  column  of  Table 
2.6),  we  obtained  across-disease  item-level  kappa  scores  averaging  0.78.  Overall,  the  10th 
percentile  of  the  distribution  of  our  kappa  scores  was  0.63,  the  90th  percentile  was  1.0,  5.6 
percent  of  our  kappa  scores  were  below  0.4  (a  level  generally  recognized  as  signifying  poor 
reliability  (Landis  and  Koch,  1977)),  and  83  percent  were  above  0.75  (a  level  implying  excel- 
lent reliability). 

Effects  Detectable  with  the  Study's  Sampling  Resources 

Table  2.7  presents  typical  examples  of  pre-  and  post-PPS  effects  detectable  with  the 
study's  resources.  It  can  be  seen  from  the  table  that  quite  small  disease-specific  and  aggre- 
gate differences  in  the  average  quality  of  processes  of  care,  and  a  modest  difference  in  aggre- 
gate mortality  across  all  five  diseases,  stand  a  high  chance  of  being  found,  whereas  disease- 
specific  differences  in  mortality  would  have  to  be  fairly  large  to  be  detectable  with  high 
likelihood.  The  estimates  in  the  table  take  into  account  both  the  potential  information  gain 
from  the  stratification  and  longitudinal  hospital  sampling  built  into  the  sampling  plan  and 
the  potential  information  loss  from  the  plan's  clustering,  and  are  typical  of  the  power  conclu- 
sions to  be  drawn  about  the  many  quantitative  and  qualitative  response  variables  in  our 
study:  Our  data  resources  were  sufficient  to  find  quite  small  disease-specific  differences  for 
quantitative  outcomes  and  moderate  differences  aggregating  across  diseases  for  dichotomous 
outcomes. 
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Table  2.6 

INTERRATER  RELIABILITY  RESULTS  FOR  SELECTED  ITEMS,  BY  DISEASE 


Gold  Standard  #la 

Gold  Standard  #2a 

Field  Datab 

Abstraction  Form  Items 

%  Agreement 

Kappa 

%  Agreement 

Kappa 

%  Agreement 

Kappa 

Congestive  Heart  Failure 

Presence  of  CHF  symptoms  at  admission  (e.g.,  dyspnea) 

1.00 

(c) 

1.00 

(c) 

1.00 

(d) 

Discharge  status  (e.g.,  discharged  dead) 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 

Physician's  documentation  of  physical  exam 

1.00 

1.00 

1.00 

1.00 

0.94 

0.84 

(e.g.,  presence  or  absence  of  heart  murmur) 

Nursing  care:  Frequency  of  blood  pressure 

1.00 

1.00 

0.93 

0.91 

0.89 

0.85 

readings  on  hospital  day  2 

Medications:  Use  of  loop  diuretic  on  hospital  day  1 

1.00 

1.00 

0.85 

0.35 

0.94 

0.88 

Electrocardiogram  obtained  on  hospital  day  1 

1.00 

1.00 

0.98 

0.89 

1.00 

1.00 

Digitalis  level  obtained  ever  during  the  hospitalization 

0.94 

0.85 

0.95 

0.88 

1.00 

1.00 

Acute  Myocardial  Infarction 

Physician  documentation  of  preadmission  symptoms  of  AMI 

0.98 

0.00 

1.00 

1.00 

0.97 

0.16 

Physician  documentation  of  timing  of  onset  of  symptoms  of  AMI 

0.98 

0.54 

1.00 

1.00 

0.97 

0.65 

Physician  documentation  of  presence  or  absence  of  heart  block 

0.70 

0.60 

1.00 

1.00 

0.75 

0.66 

Use  of  coronary  care  unit,  intensive  care  unit,  telemetry,  or  wards 

1.00 

1.00 

0.78 

0.74 

0.97 

0.95 

Use  of  digitalis 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 

Location  of  AMI  by  electrocardiogram  report 

0.50 

0.17 

0.85 

0.76 

0.75 

0.63 

CPK  isoenzyme  value  on  hospital  day  1 

0.96 

0.84 

1.00 

1.00 

0.83 

0.45 

Pneumonia 

Presence  of  pneumonia  symptoms  at  admission 

1.00 

1.00 

1.00 

1.00 

1.00 

(d) 

(e.g.,  cough,  sputum,  fever) 

Patient's  Medicaid  status 

1.00 

1.00 

0.97 

0.88 

0.74 

0.20 

Prehospital  mental  status  (e.g.,  dementia) 

1.00 

1.00 

1.00 

1.00 

0.91 

0.71 

Use  of  antibiotics  before  admission 

0.98 

0.91 

1.00 

1.00 

0.91 

0.71 

First  inhospital  respiratory  rate 

0.98 

0.98 

0.97 

0.97 

0.85 

0.83 

First  inhospital  venous  bicarbonate  value 

0.93 

0.93 

0.21 

0.12 

0.74 

0.72 

Preadmission  residence  (e.g.,  home) 

0.82 

0.67 

1.00 

1.00 

0.71 

0.59 

CHF  by  chest  X-ray  on  hospital  day  1 

1.00 

1.00 

1.00 

1.00 

0.91 

0.80 

Cerebrovascular  Accident 

Presence  of  CVA  symptoms/signs  at  admission 

1.00 

1.00 

1.00 

1.00 

1.00 

(d) 

(e.g.,  visual,  sensory,  motor,  or  speech  deficit) 

Patient's  race 

1.00 

1.00 

1.00 

1.00 

0.95 

0.88 

Discharge  destination 

1.00 

1.00 

0.97 

0.96 

0.95 

0.93 

History  of  CVA  and/or  transient  ischemic  attack  before  admission 

1.00 

1.00 

0.92 

0.85 

0.86 

0.74 

Presence  of  coma  on  hospital  day  1  and/or  2 

1.00 

1.00 

0.95 

0.78 

0.86 

0.64 

Nurse  documentation  of  patient's  pupils  on  hospital  day  1 

0.91 

0.82 

1.00 

1.00 

0.81 

0.62 

Use  of  coumarin  ever  during  the  hospital  stay 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 

Presence  of  increased  intracranial  pressure 

1.00 

1.00 

1.00 

1.00 

0.95 

0.92 

on  CAT  scan  (NMR  or  MRI  scan)  of  brain 

Use  of  prothrombin  time  (PT)  or  partial  thromboplastin 

0.96 

0.90 

1.00 

1.00 

0.90 

0.80 

time  (PTT)  ever  during  hospitalization 

Hip  Fracture 

Presence  of  actual  hip  fracture  at  admission 

1.00 

1.00 

0.96 

0.00 

1.00 

(d) 

Patient's  Medicaid  status 

0.70 

0.64 

0.93 

0.91 

0.86 

0.37 

Preadmission  residence 

0.98 

0.97 

0.95 

0.94 

0.80 

0.73 

Type  of  hip  fracture  surgery 

1.00 

1.00 

1.00 

1.00 

0.97 

0.96 

Location  of  hip  fracture  from  the  operative  report 

0.98 

0.61 

1.00 

1.00 

1.00 

1.00 

Number  of  physical  therapy  sessions  on  postoperative  day  1 

0.98 

0.98 

0.96 

0.75 

1.00 

1.00 

Use  of  parenteral  antibiotics  on  day  of  surgery 

0.80 

0.09 

0.95 

0.78 

0.91 

0.85 

Radiographic  evidence  for  osteoporosis  of  fractured  hip 

0.53 

0.00 

0.51 

0.00 

0.57 

0.33 

First  serum  albumin  value  on  hospital  day  1  or  2 

0.96 

0.96 

1.00 

1.00 

0.86 

0.82 

Gold-standard  #1  and  #2  records  were  disease-specific  records,  known  by  the  data  collectors  to  be  test  records,  against  which  the  data 
collectors'  abstraction  form  responses  were  tested.  #2  records  differed  from  #1  records  in  that  #2  records  were  compared  with  known  gold 
standards  only  after  supervisory  nurses  and  physicians  had  checked  the  accuracy  of  the  abstraction  form  data. 

Field  data  records  are  pairs  of  abstraction  forms  written  by  two  data  collectors  abstracting  the  same  medical  record  under  field  condi- 
tions, i.e.,  the  data  collectors  did  not  know  that  their  performance  was  being  monitored. 

For  gold-standard  #1  and  #2  records,  kappa  scores  were  not  calculated  if  all  abstraction  forms  studied  had  100  percent  agreement  for 
the  |em  and  there  was  no  variation  in  the  item  response. 

For  field  data  records,  kappa  scores  were  not  calculated  if  there  was  no  variability  in  the  item  response  across  the  pairs  of  study 
cases. 
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Table  2.7 

PRE-  AND  POST-PPS  DIFFERENCES  IN  30-DAY  MORTALITY, 
ADJUSTED  FOR  SICKNESS  AT  ADMISSION 
(In  percent) 


Pre-Post 

Standard  Error  for 

Pre-PPS 

Change  Detectable 

Estimated  Pre-Post 

Disease 

Rate 

with  80%  Power 

Difference 

Congestive  heart  failure 

14.7 

3.4 

1.3 

Acute  myocardial  infarction 

24.4 

4.2 

1.5 

Pneumonia 

15.9 

3.6 

1.3 

Cerebrovascular  accident 

21.3 

4.0 

1.4 

Hip  fracture 

5.1 

2.1 

0.8 

Aggregates 

16.7 

1.6 

0.6 

NOTE:  The  study's  power  to  find  1.1  point  or  greater  disease-specific  pre- 
and  post-PPS  differences  on  a  100-point  process  scale  with  patient-level  stan- 
dard deviation  10  is  at  least  80  percent.  In  the  aggregate  across  diseases,  0.5 
point  or  larger  differences  on  such  a  scale  would  be  detectable  with  at  least  80 
percent  power. 


Effects  smaller  than  those  referred  to  in  Table  2.7  as  detectable  with  80  percent  power 
may  actually  be  found  to  be  significant  by  the  study,  as  can  be  seen  from  the  column  giving 
standard  errors  for  the  estimated  pre-  and  post-PPS  differences  in  30-day  mortality,  adjusted 
for  sickness  at  admission. 

Representativeness  of  Final  Sample 

Tables  2.8  and  2.1  compare  our  sample  with  known  national  prevalences  on  a  number  of 
important  hospital  characteristics.  It  is  evident  that  after  reweighting,  the  sample  matches 
the  nation  closely  with  respect  to  hospital  size,  city  size,  percentages  of  Medicare  and  Medic- 
aid patients,  teaching  intensity,  and  hospital  ownership. 

Figure  2.1  compares  our  sampled  age-  and  sex-adjusted  30-day  mortality  values  with 
known  national  values  on  a  quarterly  basis,  in  the  aggregate  across  diseases;  the  results 
were  similar  at  the  disease-specific  level.  The  national  values  fell  within  the  sample  95  per- 
cent confidence  limits  in  11  out  of  the  12  quarters,  and  the  pattern  of  positive  and  negative 
deviations  of  the  sampled  values  from  the  national  mortality  figures  reveals  no  bias.  Both  of 
these  observations  are  consistent  with  the  hypothesis  that  our  sample  is  representative  of 
the  nation  with  respect  to  30-day  mortality. 

Composition  of  Final  Patient  Sample 

Table  2.9  gives  various  characteristics  of  the  study's  patient  sample,  both  before  and 
after  PPS,  by  disease  and  in  the  aggregate  across  diseases.  Forty-one  percent  of  our  patients 
were  over  80  years  old;  43  percent  were  male;  19  percent  were  nonwhite;  12  percent  had  a  do 
not  resuscitate  (DNR)  order  written  at  some  point  during  the  hospitalization;  15  percent  had 
Medicaid  status;  and  14  percent  were  admitted  from  a  nursing  home.  Demographics 
remained  quite  stable  from  pre-  to  post-PPS,  but  the  fraction  of  patients  with  a  DNR  order 
rose  sharply  and  significantly  for  acute  myocardial  infarction,  congestive  heart  failure,  pneu- 
monia, and  overall,  in  the  aggregate  from  10.5  percent  pre-PPS  to  13.7  percent  post-PPS  (p  < 
0.01),  a  30  percent  increase.  (See  App.  A  for  further  details  about  the  demographic  charac- 
teristics of  study  patients.) 


Table  2.8 


THE  SAMPLE  COMPARED  WITH  THE  NATION 
(In  percent) 


Raw 

Reweighted 

National 

Sample 

Sample 

Category 

Prevalence 

Prevalence 

Prevalence 

Percentage  of  Medicare  Patients  Treated  at  the  Given  Hospital3 

0-28 

25 

27 

25 

29-33 

25 

25 

26 

34-39 

25 

19 

24 

40-47 

15 

16 

16 

48+ 

10 

12 

10 

100 

100 

100 

Percentage  of  Medicaid  Patients  Treated  at  the  Given  Hospital 

0-3.8 

25 

24 

25 

3.9-7.3 

25 

16 

24 

7.4-10.8 

25 

23 

25 

10.9-16.7 

15 

18 

16 

16.8+ 

10 

19 

11 

100 

100 

100 

Ratio  of  Interns  and  Residents  to  Beds 

0 

65 

65 

65 

0-0.274 

25 

26 

25 

0.274+ 

10 

9 

10 

100 

100 

100 

1981  Standard  Metropolitan  Statistical  Area  (SMSA)  Status 

Rural 

21 

22 

21 

Up  to  250,000 

13 

10 

11 

250,000-1,000,000 

26 

23 

26 

1,000,000-2,500,000 

22 

25 

24 

2,500,000+ 

18 

21 

18 

100 

100 

100 

Hospital  Ownership  (Profit  Status,  Church  Affiliation,  etc.) 

Church  voluntary/nonprofit 

22 

22 

22 

Other  voluntary/nonprofit 

46 

39 

45 

Proprietary 

11 

12 

12 

County 

8 

14 

8.4 

State,  city,  city/county 

5 

4.3 

4.0 

Hospital  district 

8 

9.0 

9.2 

100 

100 

100 

aThese  figures  are  at  the  patient  level,  i.e.,  25  percent  is  the 
fraction  of  patients  treated  nationally  at  hospitals  with  percentages 
of  Medicare  patients  between  0  percent  and  28  percent,  not  the 
percentage  of  such  hospitals  nationwide.  National  data  are  from 
AHA  (1984)  and  HCFA  (1986b). 
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Fig.  2.1 — Quarterly  30-day  mortality  trend  for  five  diseases 


Table  2.9 

CHARACTERISTICS  OF  THE  STUDY  SAMPLE,  BY  DISEASE  AND  TIME  PERIOD1 

(In  percent) 


Patient 

CHF 

AMI 

PNE 

CVA 

HIP 

Total 

Characteristic 

Pre 

Post 

Pre 

Post 

Pre 

Post 

Pre 

Post 

Pre 

Post 

Pre 

Post 

Total 

80  or  more  years  old 

41 

41 

25 

27 

41 

42 

41 

41 

58 

58 

41 

42 

41 

Male 

43 

45 

53 

50 

51 

51 

45 

44 

21 

23 

43 

43 

43 

Nonwhite 

23 

22 

18 

16 

19 

20 

22 

20 

14 

13 

19 

18 

19 

Medicaid 

16 

15 

12 

12 

19 

19 

14 

13 

17 

15 

15 

15 

15 

Preadmission 

residence 

nursing  home 

8 

9 

4 

5 

22* 

26* 

12 

11 

24* 

20* 

14 

14 

14 

Sample  size 

1,359 

1,465 

1,416 

1,437 

1,341 

1,408 

1,382 

1,442 

1,358 

1,404 

6,856 

7,156 

14,012 

NOTE:  *  Pre-  and  post-PPS  difference  is  significant  (p  <  0.01). 

aResults  were  reweighted  to  achieve  national  representativeness  (see  the  text). 


III.  CHANGES  IN  SICKNESS  AT  ADMISSION  FOLLOWING  THE 

INTRODUCTION  OF  PPS 


Emmett  B.  Keeler,  Ph.D.,  Katherine  L.  Kahn,  M.D., 
David  Draper,  Ph.D.,  Marjorie  J.  Sherwood,  M.D., 
Lisa  V.  Rubenstein,  M.D.,  MSPH,  Ellen  J.  Reinisch,  M.S., 
Jacqueline  Kosecoff,  Ph.D.,  Robert  H.  Brook,  M.D. 


The  introduction  in  1983  of  Medicare's  Prospective  Payment  System  has  raised  a 
number  of  clinical  and  policy  questions.  Did  the  change  in  financial  incentives  from  a  cost- 
plus  reimbursement  system  to  a  fixed-price  system  based  on  DRGs  result  in  less  sick  patients 
being  admitted  to  the  hospital?  Did  mortality  within  30  or  180  days  following  hospitaliza- 
tion, after  adjusting  for  sickness  at  admission,  change  when  prospective  payment  was  intro- 
duced? Measuring  how  sick  a  patient  is  at  the  time  of  hospital  admission  is  a  prerequisite  for 
answering  such  questions.  In  this  section  we  present  measures  of  sickness  at  admission  that 
can  be  used  to  answer  these  two  questions  for  five  diseases:  congestive  heart  failure,  acute 
myocardial  infarction,  pneumonia,  cerebrovascular  disease,  and  hip  fracture.  These  mea- 
sures can  also  be  used  to  aid  clinical  decisions  at  the  individual  patient  level  and  to  help  hos- 
pitals monitor  their  outcomes.  After  describing  the  measures,  we  present  data  showing  how 
sickness  at  admission  changed  after  the  introduction  of  prospective  payment. 

In  developing  our  measures  of  sickness  at  admission,  we  paid  special  attention  to  pre- 
viously published  severity  measures,  including  those  developed  to  predict  death  for  ICU 
patients  (Knaus  et  al.,  1985;  Kruse  et  al.,  1988;  and  Lemeshow  et  al.,  1985)  and  measures 
developed  for  specific  diseases  (Killip  and  Kimball,  1967;  Norris  et  al.,  1969;  Goldman  et  al., 
1977;  and  Daley  et  al.,  1988). 

METHODS 

Variables  Measuring  Sickness  at  Admission 

We  used  literature  review,  clinical  judgment,  and  disease-specific  consensus  panels  to 
identify  variables  that  have  been  considered  important  clinical  predictors  of  either  inhospital 
death,  death  within  30  or  180  days  of  hospitalization,  nonfatal  complications,  readmissions, 
or  prolonged  nursing  home  residence.  Variables  were  included  if  (1)  they  accurately 
described  the  patient's  status  at  the  time  of  hospital  admission,  (2)  they  were  frequent 
enough  to  be  worth  collecting,  and  (3)  they  were  reliably  recorded  in  the  medical  record 
(Kahn  et  al.,  1988a).  (Further  details  about  the  development  and  collection  of  sickness  at 
admission  variables  are  presented  in  Appendix  B.) 

We  used  disease-specific  abstraction  forms  to  collect  data  about  sickness  at  admission 
from  the  medical  records  of  hospitalized  patients  (Kahn  et  al,  1988b;  Kosecoff  et  al.,  1988; 
Roth  et  al,  1988;  Rubenstein  et  al.,  1988;  and  Sherwood  et  al.,  1988).  We  included  measures 
of  patient  sickness  at  admission  based  on  the  reason  for  the  hospital  admission  (morbid  con- 
ditions) and  on  other  conditions  that  might  influence  outcomes  (comorbid  conditions).  For 
each  patient,  we  collected  data  about  acute  and  chronic  morbid  and  comorbid  diseases, 
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function,  the  number  of  body  systems  with  pathology,  and  the  APACHE  II  Acute  Physiologi- 
cal Score  (APS)  variables  (Knaus  et  al.,  1985).  For  patients  hospitalized  with  acute  myocar- 
dial infarction,  we  also  collected  data  used  to  score  the  Norris  Coronary  Prognostic  Index  and 
the  Killip  Scale  (Killip  and  Kimball,  1967;  and  Norris  et  al.,  1969).  For  hip  fracture  patients, 
we  collected  data  for  the  Goldman  Preoperative  Risk  Index  (Goldman  et  al.,  1977).  Table  3.1 
lists  variables  collected  for  all  five  study  diseases,  as  well  as  those  variables  collected 
specifically  for  each  disease. 

Fifty-two  data  collectors  (nurses  or  medical  records  technicians)  with  prior  experience 
in  medical  record  abstraction  collected  the  data.  We  document  data  availability  and  inter- 
rater  reliability  in  Sec.  II.  Abstractors  collected  data  about  sickness  at  admission  from  physi- 
cians' and  nurses'  notes  (including  emergency  room  notes),  laboratory  results,  and  radio- 
graphic reports.  In  all  instances,  we  collected  the  first  available  data  for  the  patient  after 
admission.  If  the  patient  had  missing  laboratory  or  radiographic  data  on  day  1,  we  accepted 
data  from  day  2  because  day  2  values  may  represent  the  admission  status  of  patients  admit- 
ted late  at  night. 

Analytic  Methods 

Modeling  decisions  (scaling  of  important  variables  and  subsequent  variable  selection) 
were  based  on  a  two-thirds  random  sample  of  the  patients  with  each  disease.  We  validated 
our  modeling  choices  by  repeating  the  modeling  effort  on  the  other  third  of  the  cases  and  on 
five  bootstrap  (Altman  and  Anderson,  1989)  replicates  of  the  datasets.  Final  weights 
(coefficients)  were  estimated  on  the  full  sample.  The  weights  for  the  variables  that  were  used 
to  predict  two  outcomes  are  given  here:  death  within  30  and  180  days  after  admission. 

Our  goal  was  to  create  simple  disease-specific  scales  of  the  risk  of  dying  that  were  simi- 
lar in  form  to  the  APACHE  APS  scales.  We  categorized  and  scaled  all  but  five  variables 
using  our  own  clinical  judgment  and  the  APACHE  item  scaling.  For  example,  for  patients 
with  congestive  heart  failure,  serum  AST  (SGOT)  from  40-79  U/L  was  scored  as  2  points,  and 
AST  greater  than  80  U/L  was  scored  as  4  points.  For  all  variables,  normal  and  missing 
values  received  a  score  of  zero,  and  the  maximum  score  is  assigned  to  the  sickest  people.  For 
variables  defining  the  patients'  medical  and  functional  pathology,  we  assumed  that  patients 
did  not  have  a  condition  unless  it  was  mentioned  in  their  medical  record.  The  scoring  rules 
for  all  variables  used  in  the  final  scales  are  shown  in  Table  3.2. 

In  initial  analyses,  five  variables  (systolic  blood  pressure,  age,  blood  urea  nitrogen 
(BUN),  coma,  and  temperature)  predicted  inhospital  mortality  especially  well.  We  rescaled 
these  variables,  based  on  average  mortality  at  different  levels  of  the  variable.  We  scored  age, 
BUN,  and  temperature  by  simply  subtracting  the  mean  value,  across  all  patients  with  the 
disease,  from  the  patient's  value.  Higher  initial  systolic  blood  pressure  was  a  good  sign  for 
all  diseases  but  cerebrovascular  accident.  We  scaled  it  based  on  its  effect  on  inhospital  mor- 
tality for  patients  with  congestive  heart  failure,  and  the  scale  shown  in  Table  3.2  held  up  well 
for  all  diseases.  The  relative  impact  of  confusion  (altered  mental  or  neurologic  status)  and 
coma  on  death  varied  from  disease  to  disease,  and  we  did  a  more  clinically  detailed  scaling  of 
neurologic  status  for  cerebrovascular  accident,  where  it  was  overwhelmingly  the  strongest 
predictor  of  death. 
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Table  3.2 

SCORING  FOR  SICKNESS-AT-ADMISSION  VARIABLES  THAT  PREDICT  DEATH  WITHIN 

30  AND  180  DAYS  AFTER  ADMISSION11 


Maximum 

Achievable 

Variable 

c™%.*»b 
score 

Definition 

APACHE  II  APS 

53 

Sum  of  13  scales  assessing  patient's  vital  signs,  blood  studies,  and  mental  status 

(CHF,  AMI,  PNE,  HIP)C 

APACHE  CHEd 

1 

Severe  chronic  failure  of  liver,  cardiovascular,  respiratory  or  renal  systems,  or 

immunocompromised  =  1  (CHF,  CVA,  HIP) 

Age  (standardized) 

— 

(Age-mean  sample  age  of  78  for  CHF  and  PNE,  75  for  AMI,  77  for  CVA,  81  for 

HIPVyears 

Systolic  blood  pressure 

5 

<  90  mm  Hg  =  5;  90-99  mm  Hg  =  4;  110-119  mm  Hg  =  3;  120-139  mm  Hg  =  2;  140-159 

mm  Hg  =  1;  160+  mm  Hg  =  0  (5  diseases) 

CHF  by  chest  X-ray 

3 

Severe  CHF  by  X-ray  =  3  (PNE,  CVA,  HIP);  moderate  CHF  by  X-ray  =  2  (CVA,  HIP), 

= 

1  (PNE);  mild  CHF  by  X-ray  =  1  (PNE,  CVA,  HIP);  pleural  effusion  by  X-ray  =  1  (CHF) 

BUN  (standardized) 

— 

(BUN— mean  sample  BUN  of  10  mmol/L  CHF,  8  AMI,  9.5  PNE)/7  mmol/L  (CHF,  AMI, 

PNE) 

APACHE  II  coma  scoree 

9 

Coma  =  9  (CHF,  PNE);  confusion  or  neurologic  change  =  5  (CHF,  PNE,  HIP) 

CVA  coma  score 

5 

Posturing/no  response  to  pain  =  5;  coma  by  both  physician  and  nurse  =  4;  coma  by  physician 

or  nurse  =  3;  neurologic  change  or  unable  to  follow  commands  =  2;  confusion  =  1  (CVA) 

From  nursing  home 

1 

Prior  residence  in  skilled  nursing  facility,  intermediate  care  facility,  extended  care  facility, 

or  nursing  home  with  unspecified  type  =  1  (CHF,  AMI,  CVA,  HIP) 

Mean  blood  pressuree 

4 

Mean  blood  pressure:  s  160  or  s  49  mm  Hg  =  4;  130-159  mm  Hg  =  3;  110-129  mm  Hg  or 

50-69  mm  Hg  =  2  (CHF,  CVA) 

Heart  ratee 

4 

First  heart  rate  value:  a  180  or  s  39  beats/minute  =  4;  140-179  or  40-54  beats/minute  =  3; 

110-139  or  55-69  beats/minute  =  2  (AMI) 

Serum  sodiume 

4 

First  serum  sodium  value:  a  180  or  s  110  mmol/L  =  4;  160-179  or  111-119  mmol/1  =  3; 

155-159  or  120-129  mmol/1  =  2;  150-154  mmol/L  =  1  (CHF) 

Serum  creatinine6 

4 

First  serum  creatinine  value:  £  300  \x  mol/L  =  4;  180-300  u  mol/L  =  3;  133-168  or  <  53  u 

mol/L  =  2  (HIP) 

Oxygenation6 

4 

Partial  pressure  of  oxygen:  p02  <  55  or  A-aD02  a  500  =  4;  p02  =  55-60  or  A-aD02  = 

350-499  =  3;  A-aD02  =  200-349  =  2;  p02  =  61-70  =  1  (AMI,  HIP) 

Hematocrit6 

4 

Serum  hematocrit:  *  0.60  or  <  0.20  =  4;  0.50-0.60  or  0.20-0.30  =  2;  0.46-0.50  =  1  (CHF) 

White  blood  cell  counte 

4 

Serum  white  blood  cell  count:  *  40  or  <  1  x  107L  =  4;  20-39.9  or  1  -  2.9  x  107L  =  2; 

15-19.9  x  107L  =  1  (CVA) 

Digitalis  toxicity 

4 

Toxic  digitalis  level:  >  2.8  nmol/L  =  4  (CHF) 

Serum  AST  (SGOT) 

4 

Serum  AST  (SGOT)  value:  a  80  U/L  =  4  (CHF),  2  (AMI);  40-79  U/L  =  2  (CHF),  0  (AMI) 

Creatinine  (CK)  score 

4 

CK  value:  >  3  times  laboratory's  highest  normal  value  =  4;  <  3  but  >  1.3  times  laboratory's 

highest  normal  value  =  2  (AMI) 

Serum  albumin 

4 

Serum  albumin  value:  <  25  g/L  =  4  (CVA,  HIP);  =  3  (PNE);  25-30  g/L  =  2  (PNE,  CVA, 

HIP);  >  30  g/L  or  not  tested,  but  patient  described  as  malnourished  =  3  (HIP,  PNE) 

Male 

1 

Male  =  1  (CHF,  HIP) 

Noncompliance 

1 

Noncompliance  with  medications  or  diet  =  1  (CHF) 

Use  of  ventricular  depressant 

1 

Use  of  ventricular  depressants  (e.g.,  verapamil,  disopyramide,  clonidine)  =  1  (CHF) 

Antiarrhythmic  medications 

1 

Use  of  antiarrhythmic  medication  =  1  (CHF) 

Difficulty  with  limbs 

1 

Difficulty  with  arm  and/or  leg  function  (CHF) 

Prior  hospitalization 

1 

Hospitalization  within  6  months  =  1  (CHF,  PNE) 

Location  of  AMI 

2 

Location  of  AMI:  anterior  =  2,  inferior  or  transmural  but  unspecified  =  1;  subendocardial  = 

0(AMI) 

Cardiac  disease1 

8.7 

Age  >  80  =  3.9;  age  70-79  =  3.1;  age  <  70  =  2.3;  cardiomegaly  by  X-ray  =  1.1;  severe 

CHF  by  X-ray  =  3.3;  moderate  CHF  by  X-ray  =  2;  mild  CHF  by  X-ray  or  lungs  not  clear 

on  exam  =  1;  prior  angina  =  0.4  (AMI) 

Prior  CHF 

1 

History  of  CHF  or  use  of  digitalis  at  time  of  admission  =  1  (AMI) 

Prior  valve  disease 

1 

History  of  mitral,  tricuspid,  aortic,  pulmonic  valve  disease,  or  valvular  surgery  =  1  (AMI, 

CVA) 

Prior  diabetes 

1 

History  of  diabetes  =  1  (AMI,  CVA) 

Intubation 

1 

Oropharyngeal  or  nasopharyngeal  intubation  =  1  (AMI) 

Body  system  score 

12 

Count  of  number  of  body  systems  (e.g.,  cardiovascular,  respiratory)  involved  with  acute 

or  chronic  disease  (AMI) 

Infarction  type 

3 

Hemorrhagic  infarction  =  3;  lacunar  infarction  =  -1  (CVA) 

Shortness  of  breath 

1 

Dyspnea  noted  by  physician  and/or  nurse  =  1  (PNE,  CVA) 

Arrest  in  emergency  room 

1 

Cardiac  and/or  respiratory  arrest  or  intubation  in  emergency  room  =  1  (CVA) 

Recent  CHF 

1 

CHF  diagnosed  as  active  within  1  week  of  admission  =  1  (CVA) 

Malnutrition 

1 

Noted  by  physician  at  admission  =  1  (CVA) 
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Table  3.2 — continued 


Maximum 

Achievable 

Variable 

ScoreD 

Definition 

Nonambulatory  day  1 

1 

Noted  by  physician  or  nurse  as  nonambulatory  day  1  (CVA) 

Respiratory  distress 

1 

Noted  by  physician  (PNE) 

Septic  complications 

1 

Empyema,  lung  abscess,  pericarditis,  or  meningitis  present  at  admission  (PNE) 

Prior  respiratory  failure 

1 

Noted  by  physician  (PNE) 

Thoracic  disease 

1 

Neuromuscular  disease  (e.g.,  amyotrophic  lateral  sclerosis,  myasthenia  gravis)  =  1  (PNE) 

Prior  cancer  score 

3 

Poor  prognosis  cancer  or  cancer  treated  by  chemotherapy  or  radiation  =  3;  cancer  treated 

with  hormone  therapy  =  2;  cancer  with  no  evidence  of  activity  =  1  (PNE) 

New  lung  cancer 

1 

New  cancer  diagnosis  made  during  the  hospitalization  =  1  (PNE) 

Home  oxygen 

1 

Use  of  home  oxygen  before  admission  =  1  (PNE) 

Nonambulatory  preadmission 

1 

Noted  by  physician  or  nurse  as  nonambulatory  before  admission  (PNE) 

RAND  comorbidity  scale 

45 

A  weighted  sum  of  chronic  comorbid  conditions  present  at  admission  (PNE,  HIP)^ 

Temperature  (standardized) 

Temperature  -  37.6  (HIP) 

Pneumonia  by  chest  X-ray 

1 

Chest  X-ray  report:  bilateral  pneumonia  without  CHF  =  1;  unilateral  pneumonia  or  infiltrate 

=  1/2  (HIP) 

Prior  renal  failure 

1 

History  of  renal  transplant  or  chronic  dialysis,  and  creatinine  value  on  admission  a  2.0 

mg/dl  (HIP) 

Prior  cerebrovascular  accident 

1 

Cerebrovascular  accident  before  admission  =  1  (HIP) 

Prior  chronic  obstructive 

pulmonary  disease  (COPD) 

1 

COPD,  emphysema,  asthma,  or  chronic  bronchitis  before  admission  (HIP) 

aValues  are  given  only  for  variables  included  in  the  scales  that  predict  mortality  30  and  180  days  after  admission.  Laboratory  values 
are  given  in  Systeme  International  (SI)  units. 

The  maximum  achievable  score  is  the  difference  between  the  healthiest  and  sickest  categories  in  all  instances  where  the  healthiest 
category  is  scored  0.  This  is  the  case  for  all  variables  except  "cardiac  disease  score"  and  "infarction  type,"  BUN,  age,  and  temperature. 

cThe  variable  was  used  in  the  final  sickness-at-admission  score  for  the  diseases  in  parentheses. 

^CHE  is  chronic  health  evaluation  from  the  APACHE  CHE  measure. 

eA  component  of  the  APACHE  APS  score.  We  use  the  standard  APACHE  scores. 

^Age,  cardiomegaly,  pulmonary  congestion,  and  history  of  ischemia  as  defined  by  the  Norris  Coronary  Prognostic  Index. 

Sin  the  sum  (with  weights  in  parentheses)  are  16  measures  of  prior  conditions:  cancer  (3),  cirrhosis  (2),  diabetes  (1),  cerebrovascular 
accident  (2),  chronic  renal  failure  (3),  valvular  disease  or  angina  or  myocardial  infarction  or  heart  surgery  (2),  congestive  heart  failure  (1), 
arrhythmias  (2),  swallowing  disorder  (e.g.,  aspiration,  dysphagia)  (2),  use  of  nasogastric  tube  (3),  hospitalization  in  last  month  (2),  thoracic 
or  abdominal  surgery  in  last  month  (2),  disease  of  thorax  (3),  multiple  myeloma  (2),  splenectomy  (2),  dementia  (2),  and  five  measures  of 
current  problems:  smoking  (2),  alcoholism  (2),  morbid  obesity  (2),  hypoalbuminemia  or  malnourishment  (3),  and  immunocompromised  (2). 


Variable  Selection  and  Weighting 

We  screened  variables  for  the  final  regressions  according  to  a  rule  that  combined  clini- 
cal judgment  with  statistical  evidence.  Variables  were  rated  1,  2,  3,  or  4  by  clinician  raters 
for  their  expected  association  with  poor  hospital  outcomes.  Using  this  rating,  variables  were 
kept  for  possible  inclusion  in  the  scale  (a)  if  they  were  strong  predictors  of  death  (regression  t 
statistics  >  2.5  in  regressions  that  included  systolic  blood  pressure,  age,  the  APACHE  APS 
scales,  and,  in  the  case  of  cerebrovascular  accident,  the  detailed  scale  describing  neurological 
status);  (b)  if  they  were  medium  predictors  (t  >  2)  with  clinical  score  >  1;  or  (c)  if  t  >  1.5  and 
the  clinicians  expected  a  strong  relationship  with  death  (a  clinical  score  of  3  or  4). 

Finally,  we  used  logistic  regression  for  each  outcome  (e.g.,  mortality  within  30  days  of 
admission)  on  all  the  candidate  variables  to  assign  weights  to  the  variables.  The  final 
weights  were  rounded  to  the  nearest  integer  multiple  of  the  APACHE  APS  weights  (i.e., 
logistic  regression  coefficients  were  divided  by  the  coefficient  for  APACHE  APS  and 
rounded).  Insignificant  (p  >  0.05)  variables  and  six  measures  of  sickness  implausibly  associ- 
ated in  the  regressions  with  decreased  mortality  were  dropped.  (Two  variables  expected  to 
be  associated  with  relatively  low  levels  of  sickness  were  kept:  use  of  ventricular  depressants 
and  noncompliance  with  therapy  for  congestive  heart  failure  patients.)  To  estimate  the 
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predictive  ability  of  these  sickness-at-admission  variables  conservatively,  we  divided  the 
sample  into  ten  equal  parts  at  random.  The  R-squared  values  we  report  are  the  averages  of 
the  squares  of  the  simple  correlations  between  outcomes  in  each  tenth  and  predictions  based 
on  the  other  90  percent  of  patients.  Because  APACHE  APS  weights  are  predetermined  (not 
derived  from  our  dataset),  we  did  not  need  to  use  this  technique  to  determine  the  R-squared 
for  APACHE  APS. 

To  summarize  the  predictive  power  of  our  final  scales,  we  compared  the  R-squared 
values  for  our  scales  with  those  from  less  complete  measures  of  sickness  at  admission  calcu- 
lated on  our  dataset:  age  and  sex  alone;  age,  sex,  and  APACHE;  the  APACHE  variables,  but 
reweighted  for  better  predictive  performance  in  our  patient  cohorts;  and  reweighted 
APACHE  plus  the  body  system  count.  We  also  compared  our  R-squared  values  with  those 
from  the  Medicare  Mortality  Predictor  System  (MMPS)  (Daley  et  al.,  1988).  We  were  unable 
to  do  this  directly  with  our  dataset,  because  MMPS  decisions  on  inclusion  of  patients  with 
end-stage  cancer  and  use  of  information  on  DNR  orders  were  different  from  ours.  Neverthe- 
less, published  MMPS  R-squared  values  are  closely  comparable  to  ours,  because  (1)  the 
MMPS  study  sample  was  drawn  in  a  manner  similar  to  ours,  and  (2)  the  MMPS  investigators 
used  abstraction  forms  that  were  minor  variations  on  subsets  of  our  own  abstraction  instru- 
ments. 

Changes  in  Laboratory  Ordering  and  Reporting 

Over  the  period  from  1981  to  1986,  we  observed  substantial  increases  in  the  number  of 
arterial  blood  gases  ordered,  and  lesser  increases  in  the  ordering  of  venous  blood  chemistry 
tests  such  as  sodium,  potassium,  bicarbonate,  and  creatinines.  There  were  also  increases  in 
abnormal  laboratory  values,  as  well  as  increases  in  physicians'  reporting  of  comorbidities 
over  time.  We  were  concerned  that  these  differences  might  lead  to  artifactual  increases  in 
sickness  at  admission. 

To  investigate  this,  we  divided  the  sickness  items  into  three  categories:  recording- 
sensitive  (noted  in  Table  3.3),  laboratory,  and  recording-insensitive.  We  assumed,  conserva- 
tively, that  the  change  in  the  recording-insensitive  part  of  the  scale  (i.e.,  measurement  of 
vital  signs,  sex,  and  age)  represented  the  true  change  in  sickness  over  time,  and  for  analyses 
of  changes  over  time  we  reduced  the  changes  that  we  observed  in  the  laboratory  and 
recording-sensitive  components  of  the  sickness  scale  accordingly. 

For  the  laboratory  scale,  this  was  done  by  using  missing  laboratory  value  indicators 
(one  representing  standard  blood  chemistries,  which  typically  were  missing  as  a  group,  and 
one  representing  arterial  blood  gases)  in  the  regression  analyses  that  related  sickness  to  out- 
come. Our  analyses  of  missing  values  showed  that,  on  average,  patients  missing  blood  chem- 
istries died  slightly  more  often  in  the  30  days  following  admission  than  did  those  with  normal 
measured  blood  chemistries,  whereas  patients  missing  blood  gases  were  less  likely  to  die 
than  those  with  normal  measured  blood  gas  values.  Adjustments  were  made  to  conform  with 
these  findings  and  to  make  the  changes  over  time  in  the  laboratory  abnormalities  scale  pro- 
portional to  changes  in  the  age  and  vital  signs  scale.  Other  missing  value  indicators,  such  as 
missing  AST  or  bilirubin  or  no  data  on  preadmission  ambulation,  were  included  in  the  final 
regressions  (1)  if  the  corresponding  variables  (e.g.,  ambulation)  were  included  in  the  final 
scales,  and  (2)  if  patients  missing  the  variables  had  a  significantly  different  chance  of  dying 
than  did  patients  for  whom  the  variables  were  not  missing  and  normal. 


Table  3.3 


WEIGHTS  FOR  SICKNESS-AT-ADMISSION  VARIABLES  THAT  PREDICT 
DEATH  WITHIN  30  DAYS  AFTER  ADMISSION 


Variable 

Observed 
Range 

CHF 

AMI 

Weights* 
PNE 

CVA 

HIP 

APAPHP  TT  APS*3'0 

oo 

\J.O 

i  ** 
i 

1  * 

Age  standardized 

AC\ 
41) 

f\  o 

u.z 

f\  A  * 
U.4 

U.40 

U.10 

Blood  pressure  score 

5 

4*** 

3.5*** 

2* 

2* 

1* 

CHF  by  chest  X-ray 

3 

4* 

3 

3 

3* 

BUN  standardized 

9 

3** 

2 

3* 

Coma  score0 

9 

0.5 

0.77 

-0.8* 

CVA  coma  scorec 

5 

5** 

Mean  blood  pressure 

4 

1 

Heart  rate 

4 

-1 

Serum  sodium 

4 

2 

Oxygenation 

4 

2* 

Hematocrit 

4 

-0.5 

White  blood  count 

4 

4* 

Digitalis  toxicity 

4 

1.5 

Serum  SGOT 

4 

1.5* 

3.5** 

CPK  score 

4 

-1.5* 

Infarction  type 

4 

3** 

Serum  albumin 

3 

2 

1* 

Location  of  AMI 

2 

6** 

Intubationc 

1 

9* 

Body  system  count0 

6 

2* 

RhorfTiPss  of  hrpflth0 

7* 

5* 

Arrest  in  emergency  room0 

1 

9 

Recent  CHF° 

1 

12* 

Prior  diabetes0 

1 

4* 

From  nursing  home0 

1 

3 

rvcopiratory  aistress 

fi* 

Septic  complications0 

! 

20* 

Prior  respiratory  failure0 

1 

10 

Thoracic  disease0 

1 

15 

Recently  hospitalized0 

1 

4 

Nonambulatory0 

12* 

Temperature  (standardized) 

10 

-3* 

Male 

3* 

Pneumonia  by  chest  X-ray 

6 

Prior  renal  failure0 

18* 

Prior  cerebrovascular  accident0 

3* 

Prior  COPD0 

2 

1985-1986  multiplier 

0.102 

0.082 

0.082 

0.115 

0.112 

1985-1986  intercept 

-3.43 

-3.11 

-4.22 

-3.96 

-3.99 

1985-  30-day  postadmission  mortality 

0.131 

0.248 

0.170 

0.199 

0.048 

NOTE:  Significance  is  shown  by  asterisks:  No  asterisk  shows  2<|t|<3,  *  =  3<|t|< 
5,  **  =  5  <  |  t  |  <  7,  and  ***  =  |  t  |  >  7. 

aLogistic  regression  coefficients  were  rounded  to  simple  multiples  of  the  APACHE 
coefficient. 

APACHE  II  APS. 

°Recording-sensitive  items  (in  APACHE  II  APS  only  the  coma  score  is  recording- 
sensitive). 
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Finally,  because  many  of  the  recording-sensitive  items  are  quite  rare,  and  thus  subject 
to  substantial  random  variation,  we  reduced  their  effect  on  sickness  at  admission  uniformly 
by  a  factor  that  made  the  changes  over  time  in  the  sum  of  recording-sensitive  variables  pro- 
portional to  the  changes  in  vital  signs  and  age.  Table  3.4  includes  a  table  of  all  the  adjust- 
ments and  a  detailed  example  of  their  use.  To  the  extent  that  the  increase  in  abnormalities 
and  comorbidities  is  due  to  sicker  patients  rather  than  to  changed  medical  practice,  these 
adjustments  will  lead  us  to  underestimate  increases  in  sickness  at  admission  following  the 
introduction  of  prospective  payment.  A  second  test  of  recording  and  laboratory  abnormality 
creep  is  shown  in  Table  3.5.  We  tested  whether  the  relationship  between  severity  and  mor- 
tality was  stable  over  time  by  regressing  mortality  on  sickness  at  admission,  an  indicator  for 
post-PPS,  and  their  interaction.  Only  for  pneumonia  was  the  interaction  significant,  and 
even  in  that  disease  the  interaction  was  very  small  compared  to  the  main  effect  of  sickness. 
Thus,  the  adjustments  made  directly  for  recording  and  test-"creep"  appear  to  be  adequate. 

RESULTS 

Frequency  distributions  for  sickness-at-admission  variables  are  presented  in  App.  B. 

Sickness-at-Admission  Scales 

Sickness-at-admission  variables  that  were  used  in  the  final  equations  to  predict  30-  or 
180-day  postadmission  mortality  are  given  in  Table  3.3.  Table  3.4  shows  how  data  in  Table 
3.3  can  be  used  with  data  in  Table  3.2  to  predict  the  probability  of  death  within  30  days  for  a 
patient  with  a  particular  disease  receiving  average  processes  of  care  in  1985-1986.  The 
weights  in  Table  3.3  give  the  relative  importance  of  various  characteristics  in  prognosis;  for 
example,  in  pneumonia  patients,  respiratory  distress  (with  weight  6)  is  equivalent  in  prog- 
nosis to  6/0.45  =  13.3  years  difference  in  age. 

The  models  based  on  the  bootstrap  and  validation  datasets  were  similar  to  those 
presented  in  Table  3.3.  The  choice  of  variables,  especially  rare  variables,  varied  somewhat 
from  replicate  to  replicate,  but  the  predicted  death  probabilities  from  the  different  models 
were  very  similar  to  those  derived  from  Table  3.3  (average  pairwise  correlation:  0.94). 

The  numbers  at  the  bottom  of  Table  3.3  are  used  to  convert  the  sickness-at-admission 
score  for  a  patient  to  the  predicted  probability  of  death  within  30  days,  assuming  average 
process  of  care  in  the  1985-1986  sample  (Table  3.4).  The  multiplier  is  the  change  in  the  log 
odds  of  dying  from  each  one  point  change  in  the  sickness-at-admission  score,  and  the  inter- 
cept is  the  log  odds  of  dying  for  people  with  no  abnormalities. 

Blanks  in  Table  3.3  indicate  that  the  variable  did  not  add  to  the  prediction  of  death  for  a 
particular  disease.  Variables  listed  in  Table  3.1  but  not  in  Table  3.3  did  not  add  to  prediction 
for  any  of  the  five  diseases.  In  particular,  we  tested  64,  61,  82,  83,  and  74  variables,  respec- 
tively, for  congestive  heart  failure,  acute  myocardial  infarction,  cerebrovascular  accident, 
pneumonia,  and  hip  fracture,  and  eventually  included  18,  20,  12,  25,  and  22  variables,  respec- 
tively, in  the  final  disease-specific  measures  (counting  each  of  the  13  variables  in  APACHE 
APS  separately).  Many  of  the  variables  that  were  dropped  are  strong  predictors  of  death 
when  considered  by  themselves  but  are  overshadowed  by  other  correlated  variables  that  are 
more  closely  linked  to  death. 

All  variables  in  Table  3.3  are  significant  at  least  at  the  0.05  level.  Many  variables  were 
significant  at  higher  levels.  APACHE  APS  with  rescored  low  blood  pressure,  BUN,  and  coma 
were  important  in  most  diseases,  as  were  age  and  the  presence  of  CHF  by  X-ray.  The 


Table  3.4 


CALCULATION  OF  PREDICTED  30-DAY  AND  180-DAY  POSTADMISSION 
DEATH  PROBABILITIES  FOR  A  HYPOTHETICAL  PNEUMONIA  PATIENT 
RECEIVING  AVERAGE  CARE  IN  THE  1985-1986  PERIOD 


Item  Score 

Weight 

Assumed  Characteristics 

(from  Table  3.2) 

(from  Table  3.3) 

Product 

30-Day3 

Heart  rate,  blood  pressure,  potassium  and  white  blood 

cell  abnormalities  for  an  APACHE  APS  score  of  10 

10 

1.0 

10.0 

88  years  old  (88  -  (78,  the  mean  age)  =  10) 

10 

0.45 

4.5 

Systolic  blood  pressure  of  80 

5 

2.0 

10.0 

BUN  and  temperature  same  as  sample  mean 

0 

0 

Recently  hospitalized 

1 

4.0 

4.0 

No  data  about  preadmission  ambulation 

0 

0.0 

All  other  characteristics  normal 

0.0 

Sickness-at-admission  score 

28.5 

180-Dayb 

30-day  sickness-at-admission  score 

28.5 

1.0 

28.5 

88  years  old  (88  -  (78,  the  mean  age)  *  10) 

10.0 

0.15 

1.5 

Home  oxygen 

1.0 

6.0 

6.0 

All  other  characteristics  normal 

0.0 

Sickness-at-admission  score 

36.0 

Changes  in  Laboratory  Ordering  and  Recording  over 

Timec 

30-day  sickness-at-admission  score 

28.5 

1.0 

28.5 

Missing  data  about  ambulation  before  admission 

1.0 

8.0 

8.0 

Patient  was  recently  hospitalized,  and  admission  is  in 

the  post-PPS  period. 

4.0 

-0.02 

-0.08 

Adjusted  (30-day  mortality)  sickness-at-admission  score 

36.42 

aWe  use  the  item  score  from  Table  3.2  and  the  1985-1986  multiplier  and  intercept  of  0.082  and 
—4.22  (from  Table  3.3)  to  estimate  that  patient  A  has  a  13.2  percent  probability  of  dying  within  30 
days  postadmission,  as  follows: 


(28.5  x  0.082)  -  4.22  =  -1.88,  the  logit  of  dying,  and 
exp(-1.88)/[l  +  exp(-1.88)]  =  13.2  percent. 

^The  predicted  180-day  postadmission  death  probability  is  31.1  percent,  using  the  Table  3.6  multi- 
plier and  intercept  as  follows: 

(36.0  x  0.069)  -  3.28  =  -0.796,  and 
exp(-0.796)/(l  +  exp( -0.796))  =  31.1  percent. 

Item  score  is  from  Table  3.2  and  weight  is  from  Table  3.6. 

cAnalysis  of  changes  in  laboratory  ordering  and  recording  practices  showed  that  adjustments  to 
the  30-  and  180-day  sickness-at-admission  measures  should  be  made  for  comparisons  over  time.  For 
example,  for  hip  fracture  patients,  missing  creatinine  added  2  points  to  the  30-day  sickness-at- 
admission  scale,  and  missing  blood  gases  subtracted  2  points  (i.e.,  because  patients  with  missing 
blood  gases  were  healthier  on  average  than  those  with  measured  normal  blood  gases),  and  the 
recording-sensitive  part  of  sickness  at  admission  for  hip  fracture  patients  (the  coma  score  and  the 
three  history  variables)  needed  to  be  reduced  by  0.18  in  the  post-PPS  period  to  adjust  for  increases  in 
recording.  The  recording  adjustment,  here  -0.08,  applies  only  to  admissions  in  the  post-PPS  period. 
This  patient  is  assigned  a  predicted  30-day  probability  of  dying  of  19.9  percent,  using  the  Table  3.5 
multiplier  and  intercept: 

(36.42  x  0.095)  -  4.85  =  -1.39, 

and  exp(-1.39)/[  1+  exp(-1.39)]  =  19.9  percent. 

Item  score  is  from  above  and  weight  is  from  Table  3.5. 
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Table  3.5 

WEIGHTS  FOR  MISSING  VARIABLES  AND  RECORDING  ADJUSTMENTS 
USED  FOR  COMPARISONS  OVER  TIME 


Weights 


Variable 

CHF 

AMI 

PNE 

CVA 

HIP 

30-Day  Adjustments 

30-day  scale 
Missing  creatinine 
Missing  arterial  pH 
Missing  temperature 
Missing  bilirubin 
Missing  chest  X-ray 
Missing  ambulation  data 

2*** 

4 

1*** 

2 

16* 
7** 

2*** 

3 

-0.5 
8* 

1*** 
2 
-3 

3 

2 
-2 

Study  multiplier8 
Study  intercept8 
Recording  adjustment^ 

0.107 
-3.45 
0 

0.085 
-3.33 
-0.07 

0.095 
-4.85 
-0.02 

0.114 
-4.04 
-0.04 

0.121 
-4.10 
-0.18 

180-Day  Adjustments 

180-day  scale 

Missing  ambulation  data 

Missing  chest  X-ray 

2*** 
2* 

3 

2*** 
2 

1*** 

Study  multiplier8 
Study  intercept8 
Recording  adjustment 

0.087 
-2.13 
0 

0.068 
-3.33 
-0.07 

0.078 
-3.82 
-0.07 

0.091 
-3.42 
0 

0.091 
-3.26 
-0.04 

NOTE:  Significance  is  shown  by  asterisks:  *  =  3  <  |  t  |  <  5,  **  =  5<|t| 
<  7,  and  ***  =  |  t  |  >  7. 

^hese  multipliers  and  intercepts  convert  the  adjusted  sickness-at- 
admission  score  to  the  predicted  probability  of  dying  for  the  whole  study, 
assuming  average  care. 

Applies  to  all  recording-sensitive  variables  in  Table  3.3  in  the  post-PPS 
period. 

"Applies  to  all  variables  in  Table  3.6  (except  the  30-day  score)  in  the  post- 
PPS  period. 


remaining  variables  were  disease-specific,  but  some,  such  as  SGOT  and  myocardial  infarction 
location  for  myocardial  infarction  patients,  or  cerebral  infarction  type  for  cerebrovascular 
accident  patients,  were  nevertheless  quite  important  predictors.  Table  3.6  shows  the 
sickness-at-admission  variables  that  determine  180-day 

Variables  with  large  coefficients  (i.e.,  t-values  from  the  regression  between  2  and  3), 
such  as  diseases  of  the  thorax  for  pneumonia  patients,  represent  rare  conditions.  In  these 
data,  they  were  strongly  related  to  poor  prognosis,  but  they  may  not  be  as  useful  in  group 
adjustments  as  conditions  that  are  more  frequently  present  at  admission. 

Table  3.6  shows  the  sickness-at-admission  variables  that  determine  180-day  mortality. 
The  180-day  mortality  scales  are  the  sum  of  the  30-day  mortality  scales  and  additional  vari- 
ables. (Table  3.4  shows  how  predicted  180-day  mortality  is  computed.)  The  30-day  mortality 
scales  are  the  most  important  part  of  the  180-day  mortality  scales,  both  because  180-day 
mortality  includes  30-day  mortality  and  because  many  variables  that  influence  short-term 
mortality  have  prognostic  value  over  the  longer  term  as  well. 

Age,  functioning,  and  chronic  conditions  are  important  as  predictors  of  longer-term  mor- 
tality: Prior  nursing  home  residence,  age,  and  the  APACHE  chronic  health  evaluation  are 
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Table  3.6 

WEIGHTS  FOR  SICKNESS-AT-ADMISSION  VARIABLES  THAT  PREDICT 
DEATH  WITHIN  180  DAYS  AFTER  ADMISSION 


Weights' 


Variable 

Dbserved 
Range 

CHF 

AMI 

PNE 

CVA 

HIP 

oU-day  scale 

AO 

yo 

i  *** 

1 

1 

l 

i  #  +  + 
1 

Age  standardized 

40 

0.15 

0.2 

0.35* 

APACHE  CHE 

1 

4 

7 

10* 

Urir  Dy  cnest  A-ray 

Q 

o 

A 

BUN  standardized 

9 

4** 

4 

Male 

1 

2* 

3 

From  nursing  home 

1 

4 

6 

6 

6* 

Mean  blood  pressure 

4 

-1* 

Serum  creatinine 

A 

9* 

Owpprifltinn 

\_»  A  V  fc^dlQLlWll 

4 

1 

Onrnfl  <*core 

5 

0.8 

Cardiac  disease15 

6.4 

3** 

Serum  albumin 

3 

4* 

Infarction  type 

4 

-2 

Prior  hospitalization 

1 

3* 

Difficulty  with  limbs 

1 

4 

Noncompliance 

1 

-5 

Use  of  ventricular  depressants 

1 

-5 

Antiarrhythmic  medicine 

1 

4 

Prior  CHF 

6* 

Prior  valve  disease 

I 

8 

6 

Prior  diabetes 

1 

6 

Malnutrition 

1 

6 

Nonambulatory  day  1 

7 

Nonambulatory  (before  admission) 

6 

Prior  cancer  score 

4* 

New  lung  cancer 

24* 

Home  oxygen 

6 

RAND  comorbidity  scale 

16 

1* 

1* 

Prior  renal  failure 

-18 

Pneumonia  by  chest  X-ray 

-6 

1985-1986  multiplier 

0.086 

0.065 

0.069 

0.089 

0.079 

1985-1986  intercept 

-2.10 

-3.12 

-3.28 

-3.27 

-3.12 

1985-1986  180-day  postadmission 

mortality 

0.320 

0.359 

0.313 

0.342 

0.152 

NOTE:  Significance  is  shown  by  asterisks:  No  asterisk  shows  2  <  |  t  |  <  3,  *  =  3<|t 
|  <5,  **  =  5<  |  t  |  <7,  and***  =  |  t  |  >7. 

aLogistic  regression  coefficients  were  rounded  to  simple  multiples  of  the  30-day  score 
coefficient. 

Age,  cardiomegaly,  pulmonary  congestion,  and  history  of  ischemia. 


important  predictors  for  at  least  three  diseases.  Most  of  the  other  additional  variables  relate 
to  chronic  problems,  such  as  a  history  of  congestive  heart  failure,  cancer,  and  other  comorbid- 
ities. 
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Predictive  Power  of  the  Sickness  Scales 

Table  3.7  presents  conservative  estimates  of  the  predictive  power  of  the  RAND 
sickness-at-admission  measures  for  each  of  the  five  diseases  and  compares  them  to  the 
predictive  performance  of  other  measures  of  sickness  at  admission.  In  particular,  we  exam- 
ined the  APACHE  II  APS  scale,  which  uses  the  same  scoring  and  weighting  of  the  items  for 
all  diseases;  the  APACHE  variables  with  rescaled  blood  pressure,  BUN,  coma,  and  disease- 
specific  weights  (lines  3  and  4  of  Table  3.7);  and  the  scales  used  in  the  MMPS.  Age-sex 
adjustment  does  not  explain  much  of  the  variance  in  30-day  mortality  in  patients  65  and 
older.  The  addition  of  the  generic  APACHE  APS  adds  to  explanatory  power,  but  the  power  of 
the  APACHE  variables  is  increased  if  they  are  reweighted  specifically  for  each  disease.  The 
big  increases  in  prediction  for  AMI  and  CHF  are  mainly  due  to  the  increased  weight  on 
rescaled  systolic  blood  pressure.  A  simple  count  of  diseased  body  systems  added  nothing  to 
the  predictive  power  of  the  reweighted  APACHE  variables,  except  for  pneumonia,  where 
R-squared  rose  from  21  to  23  percent,  and  hip  fracture,  where  R-squared  rose  from  3  to  5  per- 
cent. The  complete  RAND  measures  offer  further  improvement.  The  RAND  measures 
explain  about  one-quarter  of  the  variance  in  30-day  mortality  at  the  patient  level  for  patients 
with  either  a  myocardial  infarction,  pneumonia,  or  cerebrovascular  accident.  These  values 
are  better  than  those  for  MMPS,  the  best  available  sickness-at-admission  scoring  system 
before  ours  for  Medicare  patients,  except  for  congestive  heart  failure. 

Table  3.8  shows  the  30-  and  180-day  postadmission  mortality  of  those  rated  healthiest 
and  sickest  by  the  RAND  sickness-at-admission  scale.  For  each  disease,  there  are  clinically 
meaningful  differences  between  the  healthiest  and  sickest  quartiles.  For  example,  of  those 
whose  admission  characteristics  put  them  in  the  healthiest  25  percent  of  cerebrovascular 
accident  patients,  3  percent  died  in  the  30  days  following  admission,  whereas  55  percent  of 
the  quartile  of  cerebrovascular  accident  patients  that  were  sickest  at  admission  died  (a  rela- 
tive risk  of  20.0;  95  percent  confidence  limits,  12.6  -  31.7).  The  180-day  prognosis  contrasts 
are  almost  as  large:  4  percent  of  pneumonia  patients  in  the  healthiest  quartile  died,  as 
opposed  to  68  percent  of  those  in  the  sickest  quartile  (a  relative  risk  of  16.5;  95  percent 
confidence  limits  11.3  -  23.9). 


Table  3.7 

PERCENTAGE  OF  VARIANCE  EXPLAINED  BY  SICKNESS-AT-ADMISSION  SCALES 
FOR  DEATHS  WITHIN  30  DAYS  AFTER  ADMISSION 


Predictor 

CHF 

AMI 

PNE 

CVA 

HIP 

Age,  sex 

1 

2 

3 

2 

1 

Age,  sex,  APACHE  II  APS 

3 

12 

18 

17 

3 

Reweighted  APACHE  II  APSa 

10 

18 

21 

26 

3 

Reweighted  APACHE  II  APS 

and  body  system  counta 
MMPSa,b 

10 

18 

23 

26 

5 

15 

14 

18 

25 

RAND  30-day  scalea 

12 

22 

26 

30 

6 

R-squared  computed  in  conservative  way  (see  the  text). 
Values  computed  on  different  but  comparable  datasets. 
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Table  3.8 


30-DAY  AND  180-DAY  POSTADMISSION  MORTALITY,  BY 
SICKNESS-AT-ADMISSION  QUARTILE  FOR  FIVE  DISEASES 

(In  percent) 


Five 

Quartilea 

CHF 

AMI 

PNE 

CVA 

HIP 

Diseases 

30-Day 

Healthiest  quartile 

5 

9 

1 

3 

1 

2 

Middle  quartiles 

9 

17 

9 

13 

4 

10 

Sickest  quartile 

32 

55 

45 

55 

10 

43 

Overall 

14 

24 

16 

21 

5 

16 

180-Dayb 

Healthiest  quartile 

11 

11 

4 

9 

3 

7 

Middle  quartiles 

29 

28 

22 

29 

14 

24 

Sickest  quartile 

61 

70 

67 

72 

34 

64 

Overall 

32 

34 

29 

35 

16 

29 

Sickness-at-admission  quartiles  were  determined  by  rank 
ordering  patients  according  to  their  sickness-at-admission 
scores.  RAND  30-day  scale  used. 

RAND  180-day  scale  used. 


Pre-  and  Post-PPS  Comparisons 

Finally,  applying  our  sickness-at-admission  measures  to  data  collected  before  and  after 
the  introduction  of  prospective  payment  indicated  that  sickness  at  admission  increased 
significantly  post-PPS  (Table  3.9).  In  this  table,  sickness  at  admission  is  expressed  as  the 
probability  of  dying  either  30  or  180  days  postadmission  for  patients  in  each  period  who 

Table  3.9 


SICKNESS  AT  ADMISSION:  PRE-PPS  (1981-1982) 
COMPARED  WITH  POST-PPS  (1985-1986) 
(In  percent) 


Expected  Mortality8 

Five 

Sickness  at  Admission 

CHF 

AMI  PNE 

CVA 

HIP 

Diseases 

30-Day  Scale 

Pre-PPS 

13.6 

23.5  14.2 

20.6 

4.6 

15.4 

Post-PPS 

13.9 

24.8  17.1 

20.6 

5.2 

16.4 

Difference  (post  -  pre) 

0.3 

1.3  2.9*** 

0.0 

0.6** 

1.0** 

180-Day  Scale 

Pre-PPS 

32.2 

33.2  26.2 

34.9 

15.6 

28.5 

Post-PPS 

32.8 

35.3  30.7 

34.7 

16.8 

30.1 

Difference  (post  -  pre) 

0.6 

2.1*  4.5*** 

-0.2 

1.2* 

1.6*** 

Sample  sizes 

2,591 

2,588  2,536 

2,565 

2,541 

12,821 

NOTE:  Significance  of  pre-  and  post-PPS  differences  is  shown  by 
asterisks:  *  =  p  <  0.05;  **  =  p  <  0.01;  ***  =  p  <  0.001. 

aExpected  mortality  based  on  sickness  at  admission,  assuming  aver- 
age inhospital  processes  of  care  over  the  entire  study. 
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received  average  processes  of  care  (as  defined  by  the  pre-PPS  and  post-PPS  samples  com- 
bined). To  facilitate  comparison  with  the  changes  in  postadmission  mortality  reported  in  Sec. 
VII,  the  sickness  measures  are  averaged  only  on  the  92.5  percent  of  patients  for  whom  we 
have  follow-up  data  on  mortality.  (Results  on  the  full  sample  are  almost  identical.)  For  all 
five  diseases  combined,  the  30-day  mortality  to  be  expected  from  sickness  at  admission  was 
1.0  percentage  points  higher  in  1985-1986  (16.4  compared  with  15.4  percent,  p  <  0.01  by  a 
t-test),  and  the  expected  180-day  mortality  was  1.6  percentage  points  higher  (30.1  compared 
with  28.5  percent,  p  <  0.001).  Pneumonia  patients  had  the  biggest  increase  over  time  in 
expected  mortality  from  sickness  at  admission  (a  rise  of  2.9  percentage  points  of  expected 
30-day  mortality  and  4.5  percentage  points  of  180-day  mortality,  p  <  0.001),  whereas  the 
expected  mortality  resulting  from  admission  sickness  for  cerebrovascular  accident  patients 
changed  over  time  only  slightly.  If  we  had  not  used  our  adjustments  for  changes  in  labora- 
tory ordering  and  recording,  both  the  post-PPS  values  and  the  increases  in  sickness  post-PPS 
in  Table  3.9  would  have  been  an  additional  0.6  percentage  points  of  expected  mortality 
higher  than  the  indicated  values  (e.g.,  1.6  percentage  points  of  expected  30-day  mortality  and 
2.2  percentage  points  of  expected  180-day  mortality  for  all  diseases  combined). 

Are  the  Effects  of  Sickness  at  Admission  Stable  over  Time? 

Although  the  medical  record,  unlike  the  ICD-9  codes  that  are  used  to  determine  the 
DRG,  is  not  used  to  determine  payments  directly,  there  are  influences  (changes  in  medical 
culture,  fear  of  litigation)  that  may  be  pushing  hospitals  to  more  thorough  recording.  We 
tested  whether  differences  in  recording  could  be  responsible  for  part  of  the  apparent 
increased  sickness  at  admission  directly  through  our  comparison  of  recording-sensitive  com- 
pared with  recording-insensitive  sickness  variables.  We  looked  at  interactions  of  the  effects 
of  sickness  at  admission  on  mortality  with  PPS  time  period  to  see  if  adjustments  for  record- 
ing went  far  enough.  The  idea  is  that  if  the  additional  sickness  coded  is  spurious,  the  "sicker" 
people  will  have  lower  mortality  in  the  post-PPS  period  than  in  the  pre-PPS  period  when 
they  were  really  sicker.  However,  there  is  an  alternative  explanation  of  any  such  effect: 
Medical  progress  may  be  concentrated  where  the  potential  gains  are  the  largest  (i.e.,  on 
patients  who  before  the  progress  were  most  likely  to  die). 

Table  3.10  gives  the  results  of  the  simplest  such  tests:  regressions  of  mortality  30  days 
postadmission  on  sickness,  post-PPS,  and  the  interaction  between  post-PPS  and  sickness. 
The  first  column  shows  the  results  of  a  test  of  whether  the  relationship  was  significantly  dif- 
ferent over  time.  Only  for  pneumonia  was  the  interaction  of  severity  and  post-PPS 
significant  with  mortality  falling  slightly  more  for  sicker  patients  than  for  healthier.  The 
next  two  columns  show  the  coefficient  of  sickness  at  admission  in  a  logistic  regression  of  30- 
day  postadmission  mortality  (the  difference  between  the  two  columns  is  the  coefficient  of  the 
interaction).  The  changes  in  coefficients  over  time  are  quite  small  as  a  proportion  of  the  con- 
tinuing size,  reflecting  the  stability  of  the  relationship  between  sickness  and  death. 

To  understand  how  this  affects  our  estimates  of  the  change  in  outcomes  over  time,  let  s 
denote  sickness  at  admission  and  suppose  simply  that 

Mortality   =   B  +  bs  in  the  pre-PPS  period,  and 
=   A  +  as  in  the  post-PPS  period 
=    (A  +  B)/2  +  (a  +  b)s/2  using  the  pooled  data. 
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Table  3.10 


EFFECTS  OF  CHANGES  IN  RELATION  BETWEEN  SICKNESS  AND 
MORTALITY  OVER  TIME 


Diseases 

tfor 
Interaction 

Logistic  Regression 
Coefficient 

Increase  in  Mortality 
Expected  from  Sickness 

Pre-PPS 

Post-PPS 

Using  Pre 

Using  Post 

Congestive  heart  failure 

-0.56 

0.110 

0.102 

0.3 

0.3 

Acute  myocardial  infarction 

-0.22 

0.086 

0.085 

1.3 

1.3 

Pneumonia 

-2.57 

0.111 

0.086 

3.3 

2.6 

Cerebrovascular  accident 

0.24 

0.113 

0.115 

0.0 

0.0 

Hip  fracture 

-1.47 

0.146 

0.109 

0.8 

0.6 

Five  diseases 

-1.64 

1.14 

0.96 

If  average  sickness  increases  by  As,  then  we  have  used  the  pooled  correction  factor  =  (As)(a  + 
b)/2  to  adjust  outcomes  for  changes  in  sickness  at  admission. 

The  last  two  columns  of  Table  3.10  show  how  different  the  results  would  be  if  we  used 
the  pre-  or  the  post-PPS  period  relationships  between  sickness  and  mortality  to  adjust.  The 
changes  are  small  either  because  sickness  did  not  change  much  (as  in  cerebrovascular 
accident)  or  the  coefficients  did  not  change  much  (as  in  acute  myocardial  infarction).  Only 
for  pneumonia  were  both  the  change  in  sickness  and  the  change  in  the  coefficients  large. 
Overall  the  changes  in  mortality  expected  from  changes  in  sickness  are  ±0.1  percent  of  those 
given  in  Table  3.7. 

Although  it  makes  no  difference  for  this  study,  we  think  that  even  if  the  effects  of  sever- 
ity had  changed  over  time,  it  is  correct  to  adjust  outcomes  for  sickness  using  a  relation 
derived  from  pooling  the  data,  as  a  compromise  between  using  either  pre-  or  post-PPS  data. 

Here  is  an  example  to  bring  out  the  issue.  Suppose  in  1986  physicians  knew  how  to 
treat  some  comorbid  problem  that  was  fatal  in  1981.  If  the  comorbid  problem  was  part  of  the 
severity  scale,  our  adjustments  will  show  this  as  better  outcomes  for  a  very  sick  group  of 
patients:  If  the  new  treatment  was  not  in  our  process  measure,  the  better  outcomes  would 
not  be  explained  by  process  (and  in  the  text  we  would  say  they  may  be  due  to  medical  prog- 
ress). This  is  an  extreme  case,  but  if  care  is  improving  over  time,  it  will  have  to  show  up  in 
some  groups  of  patients.  If  we  adjust  for  severity  in  pre-  and  post-PPS  patients  separately, 
we  cannot  say  what  happened  to  severity  or  outcomes  over  time.  This  example  shows  that 
we  must  be  careful  to  say  that  all  changes  may  not  be  due  to  PPS,  but  that  it  is  best  to  con- 
trol as  we  did. 

The  differences  between  the  estimated  effects  including  interactions  in  the  last  two 
columns  of  Table  3.11  and  the  second  row  of  Table  7.2  in  Sec.  VII  are  very  small.  The  com- 
bined effect  of  including  the  interactions  in  the  model  for  all  diseases  was  to  increase  the 
difference  from  pre-  to  post-PPS  30-day  mortality  from  the  1  percent  we  report  in  Sec.  VII  to 
1.1  percent.  However,  because  of  the  difficulties  in  explaining  (and  understanding)  results 
including  interactions,  it  is  often  preferable  to  report  models  without  interactions,  as 
presented  elsewhere. 
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Table  3.11 

STABILITY  OF  THE  EFFECT  OF  MEASURED  SICKNESS  AT  ADMISSION  ON 
30-DAY  POSTADMISSION  MORTALITY 


Logit  Regression  Coefficients  Predicted  30-Day 


Disease 

n 

at-Admission 
Scale 

Post-PPSa 

Sickness  at 
Admission 

Interaction13 

Mortality 
Pre-PPS  Post-PPS 

Congestive  heart  failure 

2,591 

32.38 

-0.165 

0.110 

-0.007 

15.1 

13.0 

(-1.2) 

(11.06) 

(-0.56) 

Acute  myocardial  infarction 

2,588 

25.299 

0.008 

0.086 

-0.002 

24.2 

24.2 

(0.07) 

(13.88) 

(-0.21) 

Pneumonia 

2,538 

26.733 

0.212 

0.111 

-0.025 

16.4 

15.8 

(1.26) 

(14.32) 

(-2.57) 

Cerebrovascular  accident 

2,565 

19.546 

-0.14 

0.113 

0.002 

21.3 

19.9 

(-1.04) 

(15.28) 

(0.24) 

Hip  fracture 

2,541 

7.31 

-0.034 

0.146 

-0.037 

5.4 

4.6 

(-0.16) 

(7.31) 

(-1.47) 

^Numbers  in  parentheses  are  t-statistics. 

Interaction  between  post-PPS  period  and  sickness-at-admission  score. 


DISCUSSION 

Apart  from  the  MMPS  (Daley  et  al.,  1988),  previous  scales  to  measure  sickness  at 
admission  (1)  have  most  often  been  applied  to  patients  hospitalized  in  the  intensive  care  unit, 
(2)  are  based  on  data  collected  at  any  time  during  the  admission,  (3)  are  not  easily  generaliz- 
able,  or  (4)  are  based  on  variables  and  algorithms  that  are  not  in  the  public  domain.  The 
sickness-at-admission  scales  presented  here  were  developed  from  a  large  nationally  represen- 
tative sample  of  more  than  14,000  patients  with  one  of  five  diseases  at  a  representative  set  of 
297  hospitals.  They  are  publicly  available  and  could  be  used  at  either  the  individual  patient 
level  or  the  hospital  level.  For  example,  a  physician  caring  for  a  patient  with  one  of  these  five 
conditions  could  compute  our  sickness-at-admission  value  to  find  whether  the  predicted 
death  probability  for  this  patient  was  5  or  50  percent.  Hospitals  that  differ  in  their  propor- 
tion of  healthy  and  sick  patients  will  have  different  outcomes  independent  of  the  care  they 
provide;  our  scales  can  help  to  remove  this  confounding  factor  in  hospital  comparisons. 

Our  results  have  implications  for  those  who  want  to  use  APACHE  APS  or  MMPS.  We 
found  the  APACHE  II  variables  to  be  important  in  predicting  death  for  all  patients,  whether 
they  are  hospitalized  in  or  out  of  an  intensive  care  unit.  Except  for  blood  pressure  and  renal 
failure,  the  APACHE  variables  were  appropriately  scaled  and  weighted.  Low  systolic  blood 
pressure  was  an  important  predictor  of  death  for  all  five  diseases;  but  in  contrast  to 
APACHE's  result  on  intensive  care  unit  patients,  high  values  of  blood  pressure  were  more 
favorable  than  normal  values,  especially  for  patients  with  either  congestive  heart  failure  or 
myocardial  infarction,  and  reweighting  for  each  disease  led  to  much  better  predictions.  BUN 
was  important  even  after  accounting  for  serum  creatinine  values  (APACHE  measures  renal 
failure  primarily  by  creatinine).  In  addition,  we  identified  additional  laboratory  findings, 
e.g.,  SGOT  and  serum  albumin,  that  were  important  predictors  for  particular  diseases. 

Our  scales  combine  the  acute  data  that  are  the  focus  of  MMPS  with  an  inventory  of 
chronic  morbid  and  comorbid  disease  markers.  By  using  additional  data  our  predictive 
results  are  usually  better  than  those  of  MMPS.  Our  results  confirm  earlier  findings  that 
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chronic  comorbidity  is  important  for  predicting  longer-term  outcomes  (Charlson  et  al.,  1987; 
and  Clemens  et  al.,  1986).  Chronic  problems  were  also  important  for  predicting  inhospital 
and  30-day  mortality,  especially  for  pneumonia  and  hip  fracture  patients. 

Unlike  MMPS,  we  did  not  adjust  for  the  assignment  of  a  DNR  order  or  for  metastatic 
cancer,  which  in  MMPS  were  strong  predictors.  We  did  not  use  DNR  at  admission  as  a  pre- 
dictor because  of  the  possibility  that  the  large  increase  in  the  use  of  DNR  orders  from  1981  to 
1986  might  have  been  due  in  part  to  new  financial  incentives.  Patients  with  known  meta- 
static cancer  were  excluded  from  our  study  to  avoid  the  need  to  develop  a  separate  set  of 
process  criteria  for  them.  We  could  have  raised  predictive  power  by  including  these  patients 
and  adjustments. 

The  decision  about  which  sickness-at-admission  measures  to  use  depends  on  the  level  of 
clinical  data  one  is  willing  to  collect:  none,  APACHE  variables,  a  few  other  important 
disease-specific  findings,  or  a  more  extensive  inventory.  Whether  it  is  worth  the  cost  to  col- 
lect additional  clinical  variables  routinely  as  part  of  the  enormous  administrative  datasets 
compiled  by  HCFA  is  an  important  area  for  further  study.  On  the  basis  of  work  reported 
here,  to  strengthen  the  mortality  models  that  HCFA  uses  for  its  annual  hospital  mortality 
reports  (HCFA,  1987),  we  would  first  add  data  on  initial  systolic  blood  pressure,  and  perhaps 
the  few  most  important  laboratory  variables. 

Our  research  demonstrates  that  the  number  of  variables  (from  the  medical  record) 
important  in  measuring  sickness  at  admission  is  limited.  We  began  with  an  average  number 
of  73  sickness  variables  per  disease,  but  our  final  models  for  sickness  measures  that  predict 
30-day  postadmission  mortality  contained  an  average  of  19.  Our  research  also  shows  the 
advantage  of  using  disease-specific  sickness-at-admission  scales:  The  predictive  power  of 
disease-specific  scales  is  noticeably  higher  than  that  of  generic  scales  (Table  3.7). 

Many  variables  that  are  individually  strongly  associated  with  death  are  not  part  of  our 
final  scales.  Such  variables  were  rejected  for  at  least  one  of  three  reasons:  redundancy,  rare- 
ness, or  slow  effects.  When  several  variables  measure  a  similar  concept,  the  scale  includes 
the  one  with  the  most  predictive  power  and  often  excludes  the  others.  For  example,  current 
laboratory  measures  of  renal  failure  overshadow  prior  renal  failure,  except  for  hip  fracture 
patients.  Some  variables  are  just  too  rare  to  have  a  statistical  impact  on  death.  For  exam- 
ple, meningitis  at  admission  is  known  to  be  dangerous  for  pneumonia  patients,  but  with  just 
two  of  the  2,749  pneumonia  patients  in  our  sample  having  this  problem,  we  cannot  estimate 
precisely  how  dangerous  it  is.  Finally,  high  diastolic  blood  pressure  could  have  deleterious 
long-range  effects,  but  even  at  180  days  after  admission,  the  long-range  harms  were  over- 
shadowed by  the  short-range  benefits  for  our  sample  of  elderly  patients. 

In  this  study,  we  had  to  face  the  problem  of  differences  in  recording  and  laboratory 
ordering  styles  among  hospitals  and  over  time.  This  must  be  faced  in  any  comparative  study 
where  such  differences  are  expected.  Each  comparison  requires  a  study  of  recording  and  lab- 
oratory ordering  practices  and  appropriate  adjustment.  We  have  provided  one  (conservative) 
way  to  do  this. 

We  have  shown  that  patients  were  sicker  in  the  period  following  implementation  of  pro- 
spective payment.  This  increased  sickness  must  be  taken  into  account  in  evaluating  the 
changes  in  posthospital-admission  death  rates  over  time  (e.g.,  if  the  processes  of  care  were 
equally  good,  the  expected  mortality  in  the  later  period  would  be  higher).  Because  hospitals, 
following  introduction  of  prospective  payment,  had  financial  incentives  to  hospitalize  less 
sick  patients,  the  apparent  increase  in  sickness  is  initially  puzzling.  We  have  adjusted  for 
increases  in  laboratory  ordering  and  recording  of  comorbidity,  so  they  cannot  explain  the 
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rise.  Age  is  significantly  higher,  and  vital  signs  at  admission  are  significantly  worse,  for 
myocardial  infarction,  pneumonia,  and  hip  fracture  patients,  indicating  that  these  patients 
are  really  sicker  at  admission. 

So  why  are  hospitalized  patients  sicker  on  average  than  they  used  to  be?  Two  explana- 
tions are  possible:  Better  paramedical  services  keep  more  sick  people  alive  to  be  hospitalized, 
and  well  people  are  being  treated  as  outpatients.  For  acute  myocardial  infarction  patients,  it 
may  reflect  more  aggressive  prehospital  treatment  that  keeps  more  sick  patients  alive  to 
reach  the  hospital.  In  general,  financial  incentives  to  increase  admission  of  less  sick  patients 
may  be  less  important  than  were  (1)  activities  of  PROs,  (2)  increased  review  of  appropriate- 
ness of  hospitalizations,  and  (3)  shifts  from  inpatient  to  outpatient  settings  for  treatment. 
Hospital  admissions  have  fallen,  despite  the  financial  incentives,  and  these  other  trends  may 
have  prevented  the  hospitalization  of  many  not-so-sick  patients,  leaving  a  sicker  population 
in  the  hospitals. 

In  summary,  we  have  developed  sickness-at-admission  measures  for  five  diseases  that 
account  for  one-third  of  all  hospital  deaths  in  the  Medicare  population.  These  measures  are 
disease-specific,  in  the  public  domain,  and  explain  about  one-quarter  of  the  variance  in  death 
at  the  patient  level  following  hospitalization  for  acute  myocardial  infarction,  pneumonia,  or 
cerebrovascular  accident.  If  more  explanatory  power  is  desired,  then  a  data  source  other 
than  the  medical  record  will  be  required  (e.g.,  data  collected  prospectively  from  patients  or 
physicians).  Our  sickness  measures  demonstrate  a  more  than  tenfold  variation  in  actual 
death  rates  between  the  upper  and  lower  quartiles  of  patients  with  each  disease,  and  we 
should  consider  how  such  information  can  be  used  to  improve  clinical  decisions.  Finally,  we 
show  that  increases  in  sickness  at  admission  accompanied  the  unexpected  drop  in  admissions 
following  the  introduction  of  prospective  payment. 
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Processes  of  care — what  we  do  to  patients — have  been  considered  an  essential  component 
of  quality  of  care  measurement  for  over  50  years  (Donabedian,  1966).  Even  if  outcomes  of 
care — what  happens  to  patients — are  the  most  meaningful  measures  of  quality  to  the  patient, 
we  will  be  unable  to  develop  clinical  methods  to  improve  outcomes  unless  we  understand  how 
processes  and  outcomes  are  related.  Assessing  quality  of  care  by  process  also  provides  some 
measurement  advantages  over  studying  outcomes  alone.  Processes  are  more  sensitive  mea- 
sures than  outcomes  because,  in  most  clinical  situations,  not  all  patients  who  experience  poor 
process  of  care  suffer  a  poor  outcome.  For  example,  the  quality  of  prescribing  would  be  more 
easily  measured  through  process  evaluations  of  the  use  of  chloramphenicol  than  by  the 
occurrence  of  rare  outcomes  such  as  aplastic  anemia  and  death  that  may  follow  its  use.  In  addi- 
tion, outcome  measures  are  more  affected  by  differences  in  patient  sickness  than  are  process 
measures. 

Process  can  be  measured  either  implicitly  (no  preset  standards  or  criteria)  or  explicitly 
(Brook  and  Appel,  1973).  In  the  past,  it  has  proven  difficult  to  demonstrate  relationships 
between  outcomes  on  the  one  hand  and  implicit  or  explicit  assessments  of  process  on  the 
other,  despite  much  important  work  in  this  field  (Wyszewianski,  1988;  McAuliffe,  1978; 
Rubenstein  et  al.,  1977;  and  Greenfield  et  al.,  1981).  In  the  absence  of  such  evidence,  interest 
in  results  of  process  analyses  has  decreased.  At  present,  despite  hundreds  of  studies  that 
examine  the  process  of  care  provided  to  hospitalized  patients  with  common  conditions,  it  is 
impossible  to  find  a  library  that  contains  disease-specific  sets  of  process  criteria,  let  alone 
validated  criteria,  by  which,  for  example,  the  process  of  care  given  to  a  patient  with  a  myo- 
cardial infarction  can  be  evaluated. 

The  purpose  of  this  section  is  twofold.  First,  we  report  on  our  attempt  to  develop  a  set 
of  validated  process  criteria  for  elderly  patients  admitted  to  the  hospital  with  one  of  five  com- 
mon conditions.  By  validated  we  mean  that  an  established  relationship  exists  in  which  pro- 
cess predicts  outcome.  Second,  we  apply  the  validated  process  criteria  to  patients  treated 
before  and  after  the  implementation  of  PPS  to  determine  if  PPS  has  been  associated  with 
changes  in  the  processes  of  care. 

METHODS 
Sample 

We  based  our  evaluation  of  the  processes  of  care  on  the  sample  described  in  Sec.  II. 
Demographic  characteristics  of  the  study  sample  and  the  details  of  the  sampling  plan  are 
also  described  in  Sec.  II. 
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Developing  Process  Criteria 

We  sought  process  measures  that  were  clinically  valid.  We  began  our  search  with  a 
literature  review  and  consultation  with  experts.  We  specifically  asked  the  experts  to  define 
process  measures  that  they  believed  made  a  difference  in  patient  outcomes.  These  measures 
were  then  presented  to  disease-specific  panels  consisting  of  5-12  generalist  and  specialist 
physicians,  who  were  selected  by  our  collaborators  in  this  study,  the  PROs  from  the  five 
states  in  which  the  study's  sampling  occurred.  Each  panel  reviewed  the  suggested  criteria  to 
decide  (1)  whether  as  a  group  they  believed  that  these  criteria  were  likely  to  be  reliably 
recorded  in  the  medical  record  in  1981  as  well  as  in  1986,  and  (2)  whether  the  criteria  made 
clinical  sense.  We  paid  particular  attention  to  physicians'  comments  suggesting  that  varia- 
tions in  recording  of  process  or  use  of  technologies  would  be  likely  to  occur  by  year,  state,  or 
hospital  type.  Process  criteria  vulnerable  to  such  variations  were  excluded  from  the  study. 

After  developing  process  measures  that  appeared  to  be  clinically  valid,  we  then  pilot- 
tested  whether  they  could  be  reliably  assessed  using  the  medical  record  as  the  database.  For 
example,  the  number  of  times  a  frail  patient  was  moved  from  the  bed  to  a  chair  was  not 
easily  measured  from  the  record  and  was  not  included  as  a  process  measure.  Similarly,  nei- 
ther the  time  it  took  for  a  nurse  to  respond  to  a  bedridden  patient's  call  nor  the  adequacy  of 
nutrition  during  the  hospital  stay  could  be  assessed  from  the  record.  On  the  other  hand, 
indications  for  use  of  the  intensive  care  unit,  medications,  and  procedures  could  be  assessed 
from  the  medical  record,  and  these  were  included.  Finally,  when  the  panel  believed  that 
there  were  several  equally  good  strategies  for  diagnosing  or  treating  a  patient,  we  designed 
the  process  criteria  to  assign  credit  if  any  of  the  strategies  were  followed.  (Appendix  C  pro- 
vides additional  material  about  the  model  used  for  developing  explicit  process  criteria.) 

Collecting  Process  Data 

We  developed  disease-specific  abstraction  forms  to  collect  the  medical  record  data.  In 
addition  to  studying  some  process  measures  that  were  applicable  to  all  diseases,  we  studied 
some  aspects  of  process  in  more  detail  for  each  disease.  To  accomplish  this  we  collected  data 
on  approximately  100  process  criteria  for  each  disease  (see  Table  4.1).  The  instruments  for 
measuring  process  have  been  published  elsewhere  (Kahn  et  al.,  1988b;  Rubenstein  et  al., 
1988;  Kosecoff  et  al.,  1988;  Sherwood  et  al.,  1988;  and  Roth  et  al,  1988). 

We  extracted  process  data  from  the  medical  record  in  a  way  that  reflected  the  temporal 
realities  of  the  hospital  stay.  We  defined  the  initial  part  of  the  hospital  stay  as  the  first  two 
days,  because  patients  admitted  late  in  the  day  may  receive  some  or  all  of  their  initial  pro- 
cess on  the  second  hospital  day.  For  all  five  study  diseases  we  collected  process  data  on  the 
first  two  hospital  days;  the  first  time  a  clinical  problem  arose  regardless  of  the  hospital  day; 
and  the  day  before  and  day  of  discharge.  For  the  four  medical  diseases,  we  also  studied  pro- 
cess of  care  on  hospital  day  3.  For  patients  with  hip  fracture  we  collected  data  on  the  days 
surrounding  surgery. 

Some  process  measures  must  be  performed  on  a  certain  day  to  be  appropriate;  other 
process  measures  can  take  place  at  any  time  during  the  hospital  stay.  For  example,  if  the 
patient  had  a  fever,  we  collected  data  about  whether  a  fever  work-up  was  performed  during 
the  72  hours  surrounding  the  first  febrile  episode  regardless  of  when,  during  the  hospital 
stay,  the  fever  occurred.  Appendixes  A-D  present  a  complete  list  of  process  criteria  for  the 
five  study  diseases. 
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Table  4.1 

COMPONENTS  OF  INHOSPITAL  PROCESS  MEASURED  FOR 
EACH  DISEASE 


Component 

CHF  AMI 

HIP 

PNE 

CVA 

Diagnostic  Services 

Phv^ician  cognitive  assessment 

X 

X 

x 

x 

x 

Nurse  cognitive  assessment 

X 

X 

X 

X 

X 

Venous  blood  tests 

X 

X 

X 

X 

X 

Arterial  blood  tests 

X 

X 

X 

X 

X 

X-ray 

X 

X 

X 

X 

X 

EKG 

X 

X 

X 

X 

X 

Microbiology 

X 

X 

X 

Pathology 

Monitoring  with  ICU  or  telemetry 

X 

X 

X 

X 

X 

Therapeutic 

Services 

Medications 

X 

X 

X 

X 

X 

Oxygen  therapy 

X 

X 

X 

X 

Ancillary  services  (e.g.,  physical, 

respiratory  therapy) 

X 

X 

X 

Therapeutic  services  (e.g.,  surgery) 

X 

X 

X 

Scoring  Process  Criteria 

Process  criteria  were  assigned  points  by  consensus  on  the  basis  of  the  judgment  of  three 
study  physicians.  The  assignment  of  points  was  made  after  a  thorough  review  of  the  litera- 
ture and  after  consultation  with  subspecialty  experts.  First,  we  identified  the  patients  to 
whom  a  given  criterion  applied.  For  example,  the  use  of  the  intensive  care  unit  for  a  patient 
with  congestive  heart  failure  was  not  considered  indicated  for  patients  who  were  not  severely 
ill  (see  Table  4.2).  Second,  we  used  clinical  judgment  to  assign  scores  (points)  to  each  process 
criterion  based  upon  how  likely  a  patient's  outcome  was  to  benefit  from  a  given  intervention. 
For  the  physician  cognitive  scales,  which  consisted  of  8-12  dichotomous  process  criteria 
(depending  on  the  disease),  all  process  criteria  were  considered  equally  likely  to  benefit 
patient  outcome.  Thus,  all  physician  cognitive  process  items  were  assigned  a  single  point. 
However,  for  other  process  scales,  points  varied  according  to  anticipated  benefit  for  the 
patient.  For  example,  use  of  the  intensive  care  unit  for  very  sick  patients  was  worth  7  points, 
whereas  use  of  the  intensive  care  unit  for  a  moderately  sick  patient  was  worth  3  points. 
Third,  the  process  scores  were  developed  to  take  account  of  the  different  clinical  value  of 
using  the  optimal  intervention,  a  suboptimal  intervention,  or  no  intervention.  Very  sick 
patients  received  7  of  7  points  for  use  of  the  intensive  care  unit,  3  of  7  points  for  use  of 
telemetry,  and  0  of  7  points  for  no  cardiac  monitoring.  The  moderately  sick  patient  was 
assigned  3  of  3  points  for  use  of  the  ICU,  2  of  3  points  for  use  of  telemetry,  and  0  of  3  points 
for  no  cardiac  monitoring. 
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Table  4.2 


SICKNESS  LEVEL  AND  USE  OF  ICU  OR  TELEMETRY  FOR 
PATIENTS  HOSPITALIZED  WITH  CHF,  1981-1986 


Level  of 

Day  3  Use 

Sickness 

Monitoring 

ol  Monitoring 

Assigned 

Applicable 

Applicable 

on  Day  3a 

Used 

(%  of  Patients) 

Points 

Points 

Points  (%) 

Very  sick 

ICU  or  CCU 

69 

7 

7 

100 

Telemetry 

12 

3 

7 

43 

None 

19 

'  0 

7 

0 

Moderately  sick 

ICU  or  CCU 

43 

3 

3 

100 

Telemetry 

19 

2 

3 

67 

None 

38 

0 

3 

0 

Mildly  sick 

ICU  or  CCU 

20 

0 

NAb 

Telemetry 

23 

0 

0 

NA 

None 

57 

0 

0 

NA 

aDay  3  level  of  sickness  is  denned  by  a  weighted  sum  of  scores  assigned  for  the 
presence  of  each  of  the  following:  shortness  of  breath  (1  point);  chest  pain  (1  point); 
confusion  (2  points);  heart  rate  a  130  (2  points);  respiratory  rate  a  30  (2  points), 
systolic  blood  pressure  <  90  (3  points);  diastolic  blood  pressure  z  105  (1  point). 
Very  sick  =  7  or  more  points  on  this  scale;  moderately  sick  =  5-6;  and  mildly  sick  = 
0-4. 

^NA  means  that  the  process  was  not  applicable  to  this  patient  given  the 
patient's  level  of  sickness. 


Process  Scales 

Using  clinical  judgment  we  grouped  process  criteria  according  to  the  component 
categories  in  Table  4.1  and  then  tested  our  groupings  by  comparing  them  to  those  suggested 
by  a  Likert  scale  model.  All  process  criteria  in  a  Likert  scale  should  measure  a  component  of 
process  common  to  all  of  the  items  on  the  scale  but  also  measure  independent  error.  The  suc- 
cess of  the  Likert  scale  is  evaluated  by  the  interitem  reliability,  which  depends  on  the  criteria 
measuring  a  common  concept,  and  the  number  of  items  in  the  scale  (see  Table  4.3  for  inter- 
item  reliabilities)  (Nunnally,  1978).  These  methods  produced  five  conceptual  process  sub- 
scales  and  one  overall  process  scale,  as  follows:  physician  cognitive;  nurse  cognitive;  techni- 
cal diagnostic;  technical  therapeutic;  monitoring  with  ICU  or  telemetry;  and  overall  process. 
The  reliability  of  the  overall  scale  is  based  on  correlations  of  subscale  scores  rather  than  on 
correlations  of  individual  subscale  items. 

In  developing  our  process-scale  measures  we  did  not  use  the  method  of  scale  construc- 
tion used  in  Sec.  III.  First,  the  relationships  between  process  items  and  outcomes  were  much 
weaker  than  the  relationships  between  process  scales  and  outcomes.  Second,  we  believe  it 
was  important  to  use  clinical  judgment  about  process  to  assure  ourselves  that  we  were 
measuring  process  not  severity  as  a  predictor  of  outcome.  If  we  had  used  regression  as  a  tool 
to  study  which  process  items  were  most  important,  we  would  likely  have  measured  severity 
along  with  process,  since  the  regression  would  be  identifying  the  strongest  predictors  of  mor- 
tality. 

The  physician  diagnostic  cognitive  subscale  measures  the  physician's  performance  as  a 
gatherer  of  data  about  the  patient's  past  medical  history  and  current  symptoms,  and  the  per- 
formance of  physical  examinations  during  the  hospital  stay.  The  nurse  diagnostic  cognitive 
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Table  4.3 
INTERITEM  RELIABILITIES 


Condition 


Reliabilities 


Condition 


Reliabilities3 


Congestive  Heart  Failure 

Overall  Process  Scale  0.50 

MD  Cognitive  Process  Scale  0.64 

MD  initial  history  0.66 

MD  initial  physical  0.50 

MD  physical  day  2  (b) 

MD  history  and  physical  day  3  (b) 

RN  Cognitive  Process  Scale  0.36 

RN  initial  examination  0.35 
RN  examination  day  3  (b) 

Technical  Diagnostic  Scale  0.56 
Use  of  chest  X-ray  and  electrocardiogram 

day  1  and/or  2  0.40 
Use  of  chest  X-ray  and  electrocardiogram  day  3  (c) 
Use  of  venous  blood  tests  (including  CPK)  and 

arterial  blood  gas  day  1  and/or  2  0.81 
Use  of  venous  blood  tests  (including  CPK)  and 

arterial  blood  gas  day  3  (c) 
Use  of  cultures  for  febrile  patients  (b) 

Technical  Therapeutic  Scale  0.63 

Use  of  oxygen  and  digitalis  therapy  day  1  and/or  2  0.16 

Use  of  oxygen  and  digitalis  therapy  day  3  (c) 

Use  of  anticoagulant  for  nonambulatory  patient  (b) 

Monitoring  with  I CU I  Telemetry  Scale  0.94 

Monitoring  with  ICU/telemetry  day  1  and/or  2  (b) 
Monitoring  with  ICU/telemetry  day  3  (b) 

Acute  Myocardial  Infarction 

Overall  Process  Scale  0.63 

MD  Cognitive  Process  Scale  0.67 
MD  initial  history 
MD  initial  physical 
MD  history  day  2 
MD  physical  day  2 
MD  history  and  physical  day  3 

RN  Cognitive  Process  Scale  0.65 
RN  initial  examination 
RN  examination  day  3 

Technical  Diagnostic  Scale  0.58 
Use  of  chest  X-ray  day  1  and/or  2 
Use  of  chest  X-ray  day  3 
Use  of  venous  blood  tests  (including  CPK)  and 

arterial  blood  gas  day  1  and/or  2 
Use  of  venous  blood  tests  (including  CPK)  and 

arterial  blood  gas  day  3 

Technical  Therapeutic  Scale  0.46 

Use  of  pressors  and  unloading  agents  day  1  and/or  2  0.44 

Use  of  anti-arrhythmics  day  1  and/or  2  0.74 

Use  of  streptokinase  day  1  0.10 

Use  of  loop  diuretics  day  1  and/or  2  0.39 

Use  of  chest  pain  medications  day  1  and/or  2  0.15 

Use  of  pacemaker  for  heart  block  day  1  and/or  2  (d) 

Use  of  pressors  and  unloading  agents  day  3  (b) 
Use  of  medication  for  high  blood  pressure 

day  1  and/or  2  (b) 

Use  of  medication  for  high  blood  pressure  day  3  (b) 

Technical  Diagnostic  Scale  0.46 

Use  of  chest  X-ray  day  1  and/or  2  0.19 

Use  of  chest  X-ray  day  3  0.41 
Use  of  venous  blood  tests  (including  CPK)  and 

arterial  blood  gas  day  1  and/or  2  (b) 
Use  of  venous  blood  tests  (including  CPK)  and 

arterial  blood  gas  day  3  0.76 

Use  of  Swan-Ganz  Catheter  day  1  and/or  2  (b) 

Monitoring  with  ICU I 'Telemetry  Scale  0.93 

Monitoring  with  ICU/telemetry  day  1  (b) 

Monitoring  with  ICU/telemetry  day  2  (b) 

Monitoring  with  ICU/telemetry  day  3  (b) 

Pneumonia 

Overall  Process  Scale  0.40 

MD  Cognitive  Process  Scale  0.32 

MD  history  day  1  0.64 

MD  physical  day  1  0.48 

MD  physical  day  2  0.39 

MD  examination  day  3  0.37 


0.61 
0.52 
(b) 
(b) 
0.43 


0.48 
0.24 


0.19 
0.41 


0.76 


0.76 


RN  Cognitive  Process  Scale  0.36 
RN  initial  examination  day  1  and/or  2 
RN  examination  day  3 
Patient  weight  if  not  unstable 

Technical  Diagnostic  Scale  0.58 
Use  of  chest  X-ray  and  electrocardiogram  day  1  and/or  2 
Use  of  chest  X-ray  and  electrocardiogram  day  3 
Use  of  venous  blood  tests  (including  CPK)  and 

arterial  blood  gas  day  1  and/or  2 
Use  of  venous  blood  tests  (including  CPK)  and 

arterial  blood  gas  day  3 
Initial  use  of  cultures  for  patients  with 

refractory  fever 

Technical  Therapeutic  Scale  0.86 
Change  antibiotics  if  no  response 
Use  antibiotics  in  a  timely  manner 

(nonimmunocompromised) 
Use  antibiotics  in  a  timely  manner 

(immunocompromised) 
Use  of  venous  blood  tests  (including  CPK)  and 

arterial  blood  gas  day  1  and/or  2 
Use  of  venous  blood  tests  (including  CPK)  and 

arterial  blood  gas  day  3 
Use  of  oxygen  therapy  day  1  and/or  2 
Use  of  oxygen  therapy  day  3 

Monitoring  with  ICU  I  Telemetry  Scale  0.91 
Monitoring  with  ICU/telemetry  day  1  and/or  2 
Monitoring  with  ICU/telemetry  day  3 

Cerebrovascular  Accident 

Overall  Process  Scale  0.56 

MD  Cognitive  Process  Scale  0.62 
MD  history  day  1 
MD  physical  day  1 
MD  physical  day  3 
MD  history  day  2 
MD  history  day  3 

RN  Cognitive  Process  Scale  0.42 
RN  initial  examination  day  1  and/or  2 
RN  functional  assessment 
RN  examination  day  3 

Technical  Diagnostic  Scale  0,55 
Use  of  chest  X-ray/electrocardiogram  day  1  and/or  2 
Use  of  venous  blood  tests  (including  CPK)  and 

arterial  blood  gas  day  1  and/or  2 
Use  of  venous  blood  tests  (including  CPK)  and 

arterial  blood  gas  day  3 
Use  of  diagnostic  neurologic  study 

(e.g.,  EEG,  CAT  scan)  day  1  and/or  2 

Monitoring  with  ICU  I  Telemetry  _Scale  0.92 
Monitoring  with  ICU/telemetry  day  1  and/or  2 
Monitoring  with  ICU/telemetry  day  3 

Hip  Fracture 

Overall  Process  Scale  0.33 

MD  Cognitive  Process  Scale  0.55 
MD  history  day  1 
MD  physical  day  1  and/or  2 

RN  Cognitive  Process  Scale  0.00 
RN  initial  examination 
RN  examination  postoperative 

Technical  Diagnostic  (d) 
Use  of  chest  X-ray  and  electrocardiogram  day  1  and/or  2 
Use  of  venous  blood  tests  (including  CPK)  and 

arterial  blood  gas  day  1  and/or  2 
Use  of  venous  blood  tests  (including  CPK)  and 

arterial  blood  gas  day  postop 
If  hip  pinning,  then  periop  hip  X-ray 
If  hip  arthroplasty,  then  periop  hip  X-ray 
Use  of  perioperative  hematocrit 
Use  of  preoperative  electrocardiogram 
Use  of  postoperative  chest  X-ray,  electrocardiogram 


Monitoring  with  ICU /Telemetry  Scale 

Monitoring  with  ICU/telemetry  postoperative 


(b) 


0.31 
(b) 

Cb) 

0.0 
0.0 

0.80 

0.94 

(b) 

(b) 

(b) 

(b) 

(b) 

(W 
(b) 
(b) 

td) 
(b) 


0.44 
0.62 
(b) 
0.66 
0.59 

0.40 
0.36 
0.00 

0.50 
0.83 
0.69 
(b) 

(b) 
(b) 


0.31 
0.48 


0.23 

(b) 

0.56 
0.77 

0.80 
(b) 
(b) 
(b) 
(b) 

0.62 

0.94 


Interitem  reliability  is  the  square  of  the  correlation  between  the  sum  of  items  and  the  underlying  conceptual  process  value  and  is  determined 


froni  the  average  correlation  between  items. 
D 


Interitem  reliability  cannot  be  calculated  because  there  is  only  one  item  in  the  scale. 
^Reliability  not  available. 
Interitem  reliability  cannot  be  computed  because  items  are  not  measured  on  the  same  patients. 
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subscale  measures  the  nurse's  performance  as  a  gatherer  of  data  about  the  patient's  func- 
tional status,  current  symptoms,  and  vital  signs  during  the  hospital  stay.  The  technical  diag- 
nostic process  subscale  measures  use  of  diagnostic  tests  (e.g.,  venous  laboratory  studies, 
arterial  blood  gases,  X-rays,  and  electrocardiograms)  that  are  indicated  given  the  patient's 
daily  burden  of  illness.  The  technical  therapeutic  process  subscale  measures  use  of  treat- 
ments (e.g.,  medication,  surgery,  and  physical  therapy)  that  are  indicated  given  the  patient's 
daily  sickness  level.  The  ICU  or  telemetry  monitoring  subscale  evaluates  the  use  of  monitor- 
ing in  patients  as  a  function  of  their  level  of  illness.  Whereas  both  the  physician  and  nurse 
cognitive  subscales  depend  somewhat  on  styles  of  documentation  in  the  medical  record,  the 
subscales  for  technical  diagnostic,  technical  therapeutic,  and  monitoring  with  ICU  or 
telemetry  are  much  less  so. 

To  produce  these  subscales,  we  had  to  combine  some  process  measures  that  were  appli- 
cable to  all  patients  with  those  that  were  applicable  to  subsets  of  patients.  Sicker  patients 
and  those  with  longer  hospital  stays  had  more  applicable  process  criteria  than  did  less  sick 
patients.  In  general,  compliance  with  criteria  that  were  applicable  only  to  sicker  patients 
was  lower  than  was  compliance  with  those  criteria  that  were  applicable  to  all  patients.  To 
avoid  a  bias  when  combining  criteria  to  form  scales,  we  standardized  all  process  criteria  to 
have  mean  0  and  standard  deviation  1.  The  overall  scale  was  constructed  using  an  average 
of  each  of  the  equally  weighted  five  conceptual  subscales  in  like  manner.  Using  this  method 
meant  that  each  patient  who  received  average  process  would  have  an  expected  process  score 
of  0  and  standard  deviation  of  1  regardless  of  the  number  of  process  score  items  that  were 
applicable  to  the  patient. 

In  summary,  the  overall  process  scale  is  constructed  by  taking  the  subscales,  standard- 
izing them,  to  mean  0  standard  deviation  1,  averaging  the  subscales,  and  restandardizing 
them  to  mean  0,  standard  deviation  1  for  everyone  in  the  sample  regardless  of  what  pattern 
of  subscale  scores  were  available  to  them. 

To  validate  our  process  scales,  we  used  logistic  regression  to  examine  the  relationship 
between  inhospital  process-scale  scores  and  mortality  30  and  180  days  after  admission,  after 
adjusting  for  disease-specific  sickness  at  admission.  We  also  used  logistic  regression  to  deter- 
mine the  association  of  PPS  with  changes  in  process.  Our  methods  for  measurement  of  and 
adjustment  for  sickness  at  admission  are  described  in  Sec.  III.  (Additional  background 
material  about  how  explicit  process  measures  were  analyzed  is  reported  in  App.  C.) 

RESULTS 

Reliability  and  Validity  of  Process  Measures 

As  viewed  by  our  physician  panels,  the  process  measures  made  clinical  sense  and  were 
representative  of  important  areas  of  inhospital  processes  of  care  (see  Tables  4.13-4.17, 
below).  (Also  see  App.  C.)  The  item-level  reliabilities  in  Table  4.3  are  for  the  most  part  accep- 
tably high  (median  0.52).  The  intercorrelation  of  the  subscales  is  about  the  same  as  the 
intercorrelation  of  the  items  within  the  subscales.  That  is,  they  are  significantly  greater 
than  0,  but  still  low  (e.g.,  0.1-0.4). 

Compliance  was  high  for  most  of  the  explicitly  stated  process  criteria  (Tables  4.13-4.17, 
below),  supporting  the  notion  of  content  validity  (Nunnally,  1978).  However,  for  21  percent 
of  the  congestive  heart  failure  patients,  16  percent  of  the  acute  myocardial  infarction 
patients,  and  24  percent  of  the  pneumonia  patients,  the  presence  or  absence  of  a  heart 
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murmur  was  not  noted  in  the  medical  record  (Apps.  A-D).  For  19  percent  of  congestive  heart 
failure  patients,  26  percent  of  pneumonia  patients,  and  17  percent  of  cerebrovascular 
accident  patients,  fewer  than  three  blood  pressure  readings  were  taken  on  day  2  of  the  hospi- 
talization. Five  percent  of  congestive  heart  failure  patients,  6  percent  of  acute  myocardial 
infarction  patients,  9  percent  of  cerebrovascular  accident  patients,  and  10  percent  of  pneu- 
monia and  hip  fracture  patients  did  not  receive  a  serum  potassium  study  on  days  1  or  2  of  the 
hospital  stay.  One-quarter  of  patients  sick  enough  to  be  hospitalized  for  congestive  heart 
failure  did  not  have  a  serum  creatinine  study  in  the  first  two  days,  whereas  one-third  of  the 
congestive  heart  failure  patients  admitted  in  a  moderately  or  very  sick  condition  did  not  have 
any  creatinine  phosphokinase  enzyme  studies  to  rule  out  an  acute  myocardial  infarction  on 
days  1  or  2  of  the  hospitalization. 

A  better  test  of  the  validity  of  process  measures,  however,  is  the  documentation  of  a 
relationship  between  processes  of  care  and  outcomes.  Toward  this  end  we  evaluated  the  rela- 
tionship between  the  process  scales  and  30-day  postadmission  mortality.  For  this  analysis, 
patients  were  ranked  according  to  their  process-scale  scores.  Patients  labeled  as  having  good 
process  were  in  the  top  quartile  (25  percent),  and  patients  labeled  as  having  poor  process 
were  in  the  bottom  quartile  (25  percent).  For  patients  hospitalized  with  congestive  heart 
failure,  acute  myocardial  infarction,  pneumonia,  and  cerebrovascular  accident,  better  process 
is  significantly  associated  with  lower  30-day  postadmission  mortality  rates.  For  example, 
Table  4.4  shows  that  mean  30-day  postadmission  mortality,  adjusted  for  sickness  at  admis- 
sion, for  patients  hospitalized  with  congestive  heart  failure,  is  11  percent  for  patients  receiv- 
ing good  overall  process,  13  percent  for  patients  receiving  medium  overall  process,  and  19 
percent  for  patients  receiving  poor  overall  process  (p  =  0.0002).  The  relative  risk  of  adjusted 
30-day  death  as  process  changed  from  good  to  poor  was  1.74  (95  percent  confidence  interval 
1.34-  2.26;  p<  0.001). 

Table  4.4 

RELATIONSHIPS  BETWEEN  30-DAY  POSTADMISSION  MORTALITY  ADJUSTED  FOR 
SICKNESS  AT  ADMISSION  AND  OVERALL  PROCESS  FOR  FIVE  DISEASES 
PRE-  AND  POST-PPS  COMBINED 
(In  percent) 

Overall  Process  Scale 
Score  Category8 


Disease 

Good 
Process 

Medium 
Process 

Poor 
Process 

p-value*3 

Relative  Risk 
as  Process  Changes 
from  Good  to  Poor 

Congestive  heart  failure 

10.7 

12.9 

18.6 

<  0.01 

1.74  (0.23)c 

Acute  myocardial  infarction 

23.9 

22.0 

30.1 

<  0.01 

1.26  (0.11) 

Pneumonia 

14.8 

15.2 

20.2 

<  0.01 

1.36  (0.16) 

Cerebrovascular  accident 

18.7 

20.3 

25.5 

<  0.01 

1.36  (0.14) 

Hip  fracture 

5.1 

5.2 

4.6 

>0.05 

0.90  (0.22) 

NOTE:  Adjustments  for  sickness  at  admission  are  made  using  the  significant  predic- 
tors of  30-day  postadmission  death  status  on  days  1  or  2  of  admission. 

aPatients  were  ranked  according  to  process  scale  scores.  Patients  with  the  highest  25 
percent  of  process  scale  scores  were  assigned  good  process.  Patients  with  the  lowest  25 
percent  of  process  scale  scores  were  assigned  poor  process.  Other  patients  were  assigned 
medium  process. 

^From  tests  of  the  significance  of  the  process  coefficients  in  the  logistic  regressions  of 
mortality  on  process  measured  continuously,  after  adjustment  for  sickness  at  admission. 
cValues  in  parentheses  are  approximate  standard  errors  for  relative  risks. 
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For  hip  fracture  patients  we  were  unable  to  demonstrate  a  process-outcome  link  when 
outcome  was  denned  as  30-day  postadmission  mortality.  This  occurred  partly  because  only  5 
percent  of  hip  fracture  patients  died  within  30  days.  The  relatively  low  mortality  rate  for  hip 
fracture  patients  limits  our  power  to  detect  a  process-outcome  relationship. 

To  ascertain  whether  the  overall  process-outcome  relationship  was  maintained  for  the 
five  conceptual  scales  that  constitute  the  composite  process  scale,  we  examined  the  relation- 
ship between  30-day  postadmission  mortality,  adjusted  for  sickness  at  admission,  and  each  of 
the  process  scales  for  each  of  the  five  diseases  (Table  4.5).  A  significant  process-outcome  rela- 
tionship in  the  expected  direction  existed  for  four  of  the  five  scales  for  congestive  heart 
failure,  acute  myocardial  infarction,  and  pneumonia,  and  for  three  of  the  four  scales  for  cere- 
brovascular accident.  In  all  instances,  we  were  unable  to  find  a  clinically  sensible  process- 
outcome  link  for  the  process  scale  based  on  monitoring  with  ICU  or  telemetry.  We  found  one 
significant  relationship  between  ICU/telemetry  process  and  30-day  postadmission  mortality 
for  hip  fracture,  but  it  was  in  the  unexpected  direction  (better  process,  worse  outcome). 
Tables  4.6  through  4.11  show  the  relationship  between  each  of  the  process  scales  and  30-day 
postadmission  mortality  stratified  by  sickness  at  admission  quartiles. 

Process  of  Care  Pre-  and  Post-PPS 

For  each  process  scale  for  the  five  diseases,  we  found  better  process  of  care  post-PPS 
than  pre-PPS.  Table  4.12  shows  the  amount  of  improvement  in  process  scores  post-PPS  as 
compared  with  pre-PPS.  In  all  instances  the  improvement  was  significant  (p  <  0.05)  except 
for  monitoring  with  intensive  care  unit  or  telemetry,  where  process  changed  significantly  (p  < 
0.05)  only  for  congestive  heart  failure  and  hip  fracture.  The  post-PPS  improvements  in  pro- 
cess were  apparent  both  for  process  measures  that  could  have  been  influenced  by  changes  in 
recording  in  the  medical  record  (e.g.,  physician  and  nurse  cognitive)  and  for  process  mea- 
sures that  were  unlikely  to  be  affected  by  such  potential  biases  (e.g.,  appropriate  use  of  tech- 
nical diagnostics  and  technical  therapeutics).  Table  4.12  also  demonstrates  the  importance  of 
the  post-PPS  improvements  in  process  of  care.  For  example,  for  congestive  heart  failure,  we 

Table  4.5 

RELATIONSHIPS  BETWEEN  30-DAY  POSTADMISSION  MORTALITY  ADJUSTED  FOR 
SICKNESS  AT  ADMISSION  AND  PROCESS  SCALES'1  FOR  FIVE  DISEASES 

(In  percent) 

CHF  AMI  PNE  CVA  HIP 


Process  Subscale  Good    Poor    Good    Poor    Good    Poor    Good   Poor    Good  Poor 


Physician  cognitive 

12 

16* 

23 

28* 

15 

19* 

18 

24* 

6 

5 

Nurse  cognitive 

11 

17* 

24 

27* 

15 

19* 

19 

24* 

4 

6 

Technical  diagnostic 

11 

16* 

24 

29* 

14 

19* 

19 

25* 

4 

5 

Technical  therapeutic 

11 

21* 

29 

21+ 

15 

21* 

(M 

(b) 

5 

5 

Monitoring  with  ICU/telemetry 

18 

13 

21 

28* 

19 

15 

23 

21 

10 

5+ 

Overall 

11 

19* 

24 

30* 

15 

20* 

19 

26* 

5 

5 

NOTE:  *  p  <  0.05  using  a  logistic  regression  of  mortality  (adjusted  for  sickness  at  admission)  on 
process.  +  paradoxical  p  <  0.05:  better  process,  worse  outcome. 

aPatients  were  ranked  according  to  process-scale  scores.  Patients  with  the  highest  25  percent  of 
process-scale  scores  were  assigned  good  process  and  patients  with  the  lowest  25  percent  of  process- 
scale  scores  were  assigned  poor  process. 

bThe  technical  therapeutics  scale  was  not  measured  for  CVA. 
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Table  4.6 

RELATIONSHIPS  BETWEEN  30-DAY  POSTADMISSION  MORTALITY  AND  OVERALL 
PROCESS  STRATIFIED  BY  SICKNESS-AT-ADMISSION  QUARTILE 

(In  percent) 


Sickness  at  Admission8 

CHF  Process15 

AMI  Process 

PNE  Process 

CVA  Process 

HIP  Process 

Poor 

Med. 

Good 

Poor 

Med. 

Good 

Poor 

Med. 

Good 

Poor 

Med. 

Good 

Poor 

Med.  Good 

Quartile  I  (least  sick) 

8 

5 

4 

17 

6 

10 

2 

1 

1 

4 

2 

2 

1 

1  1 

Quartile  II 

7 

7 

4 

18 

10 

15 

4 

5 

4 

13 

9 

7 

4 

3  3 

Quartile  III 

20 

11 

8** 

22 

21 

19 

19 

12 

14 

22 

17 

17 

5 

5  5 

Quartile  TV  (sickest) 

40 

29 

28* 

64 

51 

50** 

55 

44 

40** 

64 

54 

49** 

7 

11  11* 

NOTES:  *  =  upper  quartile  different  from  lower  quartile,  p  <  0.05;  **  =  upper  quartile  different  from  lower  quartile,  p  <  0.01; 
and  ***  =  upper  quartile  different  from  lower  quartile,  p  <  0.001. 

aSickness-at-admission  quartiles  are  denned  by  ranking  patients  using  the  sickness-at-admission  scales  designed  to  predict 
death  within  30  days  of  admission.  See  Sec.  III.  Quartile  I  includes  the  25  percent  of  the  patient  sample  with  the  least  sickness 
at  admission  (lowest  scores).  Quartile  TV  includes  the  25  percent  of  the  patient  sample  with  the  most  sickness  at  admission 
(highest  scores). 

Process  categories  are  defined  by  ranking  patients  using  the  process  scores.  The  25  percent  of  patients  with  the  lowest 
scores  are  assigned  poor  process  and  those  with  the  highest  25  percent  of  scores  are  assigned  good  process.  The  remaining  50 
percent  are  assigned  medium  process. 


Table  4.7 

RELATIONSHIPS  BETWEEN  30-DAY  POSTADMISSION  MORTALITY  AND  PHYSICIAN 
COGNITIVE  DIAGNOSTIC  STRATIFIED  BY  SICKNESS-AT-ADMISSION  QUARTILE 

(In  percent) 


CHF  Processb 

AMI  Process 

PNE  Process 

CVA  Process 

HIP  Process 

Sickness  at  Admission8 

Poor 

Med. 

Good 

Poor 

Med. 

Good 

Poor 

Med. 

Good 

Poor 

Med. 

Good 

Poor 

Med. 

Good 

Quartile  I  (least  sick) 

4 

7 

4 

12 

10 

6 

2 

1 

6 

4 

3 

1 

1 

1 

2 

Quartile  II 

8 

6 

5 

13 

12 

14 

4 

4 

4 

15 

7 

7* 

3 

4 

4 

Quartile  III 

14 

11 

12 

24 

20 

20 

18 

12 

15 

18 

18 

18 

5 

6 

4 

Quartile  IV  (sickest) 

37 

32 

26* 

62 

52 

53 

52 

48 

36** 

59 

58 

45** 

9 

7 

13 

NOTES:  *  =  upper  quartile  different  from  lower  quartile,  p  <  0.05;  **  =  upper  quartile  different  from  lower  quartile,  p  <  0.01; 
and***  =  upper  quartile  different  from  lower  quartile,  p  <  0.001. 

8Sickness-at-admission  quartiles  are  defined  by  ranking  patients  using  the  sickness-at-admission  scales  designed  to  predict 
death  within  30  days  of  admission.  See  Sec.  III.  Quartile  I  includes  the  25  percent  of  the  patient  sample  with  the  least  sickness 
at  admission  (lowest  scores).  Quartile  P/  includes  the  25  percent  of  the  patient  sample  with  the  most  sickness  at  admission 
(highest  scores). 

^Process  categories  are  defined  by  ranking  patients  using  the  process  scores.  The  25  percent  of  patients  with  the  lowest 
scores  are  assigned  poor  process  and  those  with  the  highest  25  percent  of  scores  are  assigned  good  process.  The  remaining  50 
percent  are  assigned  medium  process. 
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Table  4.8 


RELATIONSHIPS  BETWEEN  30-DAY  POSTADMISSION  MORTALITY  AND  NURSE 
COGNITIVE  DIAGNOSTIC  STRATIFIED  BY  SICKNESS-AT-ADMISSION  QUARTILE 

(In  percent) 


CHF  Process13 

AMI  Process 

PNE  Process 

CVA  Process 

HIP  Process 

Sickness  at  Admission3 

Poor 

Med. 

Good 

Poor 

Med. 

Good 

Poor 

Med. 

Good 

Poor 

Med. 

Good 

Poor 

Med. 

Good 

Quartile  I  (least  sick) 

6 

7 

2 

11 

9 

9 

1 

0 

1 

2 

3 

3 

2 

2 

1 

Quartile  II 

10 

4 

4* 

17 

12 

13 

5 

4 

4 

13 

8 

6* 

3 

5 

3 

Quartile  III 

15 

12 

8* 

22 

20 

20 

17 

13 

13 

20 

19 

14 

6 

6 

4 

Quartile  IV  (sickest) 

37 

28 

28 

59 

53 

53 

53 

44 

39** 

59 

53 

53 

13 

11 

8 

NOTES:  *  =  upper  quartile  different  from  lower  quartile,  p  <  0.05;  **  =  upper  quartile  different  from  lower  quartile,  p  <  0.01; 
and  ***  =  upper  quartile  different  from  lower  quartile,  p  <  0.001. 

Sickness-at-admission  quartiles  are  denned  by  ranking  patients  using  the  sickness-at-admission  scales  designed  to  predict 
death  within  30  days  of  admission.  See  Sec.  III.  Quartile  I  includes  the  25  percent  of  the  patient  sample  with  the  least  sickness 
at  admission  (lowest  scores).  Quartile  rV  includes  the  25  percent  of  the  patient  sample  with  the  most  sickness  at  admission 
(highest  scores). 

^Process  categories  are  defined  by  ranking  patients  using  the  process  scores.  The  25  percent  of  patients  with  the  lowest 
scores  are  assigned  poor  process  and  those  with  the  highest  25  percent  of  scores  are  assigned  good  process.  The  remaining  50 
percent  are  assigned  medium  process. 


Table  4.9 


RELATIONSHIPS  BETWEEN  30-DAY  POSTADMISSION  MORTALITY  AND  TECHNICAL 
DIAGNOSTIC  STRATIFIED  BY  SICKNESS-AT-ADMISSION  QUARTILE 

(In  percent) 


CHF  Process13 

AMI  Process 

PNE  Process 

CVA  Process 

HIP  Process 

Sickness  at  Admission6 

Poor 

Med. 

Good 

Poor 

Med. 

Good 

Poor 

Med. 

Good 

Poor 

Med. 

Good 

Poor 

Med. 

Good 

Quartile  I  (least  sick) 

6 

6 

4 

11 

7 

13 

1 

1 

1 

4 

2 

2 

1 

1 

2 

Quartile  II 

6 

6 

6 

16 

11 

15 

4 

5 

3 

10 

10 

8 

4 

4 

2 

Quartile  III 

15 

16 

9* 

25 

19 

18 

16 

15 

12 

24 

15 

17 

4 

7 

4 

Quartile  fV  (sickest) 

35 

38 

26* 

65 

53 

48*** 

53 

47 

38** 

62 

57 

47** 

12 

11 

9 

NOTES:  *  =  upper  quartile  different  from  lower  quartile,  p  <  0.05;  **  =  upper  quartile  different  from  lower  quartile,  p  <  0.01; 
and  ***  =  upper  quartile  different  from  lower  quartile,  p  <  0.001. 

aSickness-at-admission  quartiles  are  defined  by  ranking  patients  using  the  sickness-at-admission  scales  designed  to  predict 
death  within  30  days  of  admission.  See  Sec.  III.  Quartile  I  includes  the  25  percent  of  the  patient  sample  with  the  least  sickness 
at  admission  (lowest  scores).  Quartile  IV  includes  the  25  percent  of  the  patient  sample  with  the  most  sickness  at  admission 
(highest  scores). 

Process  categories  are  defined  by  ranking  patients  using  the  process  scores.  The  25  percent  of  patients  with  the  lowest 
scores  are  assigned  poor  process  and  those  with  the  highest  25  percent  of  scores  are  assigned  good  process.  The  remaining  50 
percent  are  assigned  medium  process. 
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Table  4.10 


RELATIONSHIPS  BETWEEN  30-DAY  POSTADMISSION  MORTALITY  AND  TECHNICAL 
THERAPEUTIC  STRATIFIED  BY  SICKNESS-AT-ADMISSION  QUARTILE 

(In  percent) 


CHF  Processb 

AMI  Process 

PNE  Process 

HIP  Process 

Sickness  at  Admission3 

Poor 

Med. 

Good 

Poor 

Med. 

Good 

Poor 

Med. 

Good 

Poor 

Med. 

Good 

Quartile  I  (least  sick) 

6 

6 

5 

7 

10 

13* 

1 

1 

1 

1 

1 

1 

Quartile  II 

10 

10 

5 

8 

15 

7 

5 

1* 

4 

4 

3 

Quartile  III 

20 

20 

10** 

16 

22 

26 

21 

12 

12* 

5 

5 

6 

Quartile  IV  (sickest) 

44 

44 

24*** 

54 

57 

56 

54 

43 

46 

12 

12 

8 

NOTES:  *  =  upper  quartile  different  from  lower  quartile,  p  <  0.05;  **  =  upper  quartile  different  from 
lower  quartile,  p  <  0.01;  and  ***  =  upper  quartile  different  from  lower  quartile,  p  <  0.001. 

aSickness-at-admission  quartiles  are  denned  by  ranking  patients  using  the  sickness-at-admission  scales 
designed  to  predict  death  within  30  days  of  admission.  See  Sec.  III.  Quartile  I  includes  the  25  percent  of  the 
patient  sample  with  the  least  sickness  at  admission  (lowest  scores).  Quartile  IV  includes  the  25  percent  of 
the  patient  sample  with  the  most  sickness  at  admission  (highest  scores). 

''Process  categories  are  denned  by  ranking  patients  using  the  process  scores.  The  25  percent  of  patients 
with  the  lowest  scores  are  assigned  poor  process  and  those  with  the  highest  25  percent  of  scores  are  assigned 
good  process.  The  remaining  50  percent  are  assigned  medium  process. 


Table  4.11 

RELATIONSHIPS  BETWEEN  30-DAY  POSTADMISSION  MORTALITY  AND  MONITORING 
AND  ICU  STRATIFIED  BY  SICKNESS-AT-ADMISSION  QUARTILE 

(In  percent) 


CHF  Processb 

AMI  Process 

PNE  Process 

CVA  Process 

HIP  Process 

Sickness  at  Admission8 

Poor 

Med. 

Good 

Poor 

Med. 

Good 

Poor 

Med. 

Good 

Poor 

Med. 

Good 

Poor 

Med. 

Good 

Quartile  I  (least  sick) 

5 

4 

12* 

9 

36 

9 

1 

1 

1 

2 

2 

2 

1 

1 

1 

Quartile  II 

6 

15 

6 

14 

25 

12 

3 

19 

11** 

9 

11 

21 

3 

3 

4 

Quartile  III 

12 

15 

14 

22 

48 

19 

12 

60 

20* 

17 

37 

20 

4 

18 

18** 

Quartile  IV  (sickest) 

30 

33 

40* 

66 

93 

43*** 

45 

50 

48 

55 

58 

57 

9 

23 

17 

NOTES:  *  =  upper  quartile  different  from  lower  quartile,  p  <  0.05;  **  =  upper  quartile  different  from  lower  quartile,  p  <  0.01; 
and  ***  =  upper  quartile  different  from  lower  quartile,  p  <  0.001. 

aSickness-at-admission  quartiles  are  denned  by  ranking  patients  using  the  sickness-at-admission  scales  designed  to  predict 
death  within  30  days  of  admission.  See  Sec.  III.  Quartile  I  includes  the  25  percent  of  the  patient  sample  with  the  least  sickness 
at  admission  (lowest  scores).  Quartile  TV  includes  the  25  percent  of  the  patient  sample  with  the  most  sickness  at  admission 
(highest  scores). 

^Process  categories  are  defined  by  ranking  patients  using  the  process  scores.  The  25  percent  of  patients  with  the  lowest 
scores  are  assigned  poor  process  and  those  with  the  highest  25  percent  of  scores  are  assigned  good  process.  The  remaining  50 
percent  are  assigned  medium  process. 
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can  use  the  previously  demonstrated  process-outcome  link  to  translate  the  post-PPS  gain  in 
the  physician  cognitive  process  scale  of  0.31  to  an  expected  0.5  percentage  point  reduction  in 
30-day  postadmission  mortality  and  an  expected  0.7  percentage  point  reduction  in  180-day 
postadmission  mortality  (see  Table  4.12).  For  each  disease  except  hip  fracture,  the  post-PPS 
improvements  in  the  overall  process  scale  were  associated  with  an  expected  reduction  both  in 
30-day  mortality  (ranging  from  0.1-1.4  percentage  points)  and  180-day  mortality  (with  a 
range  from  0.4-1.6  percentage  points). 

We  also  compared  the  strength  of  the  process-outcome  link  pre-PPS  compared  with 
post-PPS.  The  process-outcome  link  was  slightly  and  insignificantly  stronger  post-PPS  for 
the  overall  scale  (p  =  0.2),  physician  cognitive  scale  (p  =  0.1),  nurse  cognitive  scale  (p  =  0.1), 
and  the  technical  therapeutic  scale  (p  =  0.9).  The  link  was  slightly  and  insignificantly 
weaker  for  the  technical  therapeutic  scale  (p  =  0.9).  It  was  not  significant  in  either  PPS 
period  for  the  monitoring  with  ICU/telemetry  scale. 

These  improvements  in  the  process  scales  post-PPS  are  a  direct  result  of  improvements 
in  the  individual  process  items  that  constitute  the  scales  (Tables  4.13  through  4.17).  For 
example,  for  congestive  heart  failure  patients,  58  percent  had  a  documentation  of  a  day  2 
lung  examination  pre-PPS,  compared  with  71  percent  post-PPS  (App.  A).  For  congestive 
heart  failure,  the  fraction  of  patients  for  whom  nurses  documented  at  least  three  blood  pres- 
sure readings  on  day  2  increased  from  78  percent  pre-PPS  to  84  percent  post-PPS.  For  all 
diseases  the  use  of  the  electrocardiogram  on  admission  to  the  hospital  increased.  The  use  of 
oxygen  (or  intubation)  on  day  1  for  hypoxic  patients  (i.e.,  p02  <  60)  improved  from  87  percent 
pre-PPS  to  93  percent  post-PPS  (for  congestive  heart  failure).  Antibiotics  for  pneumonia 
patients  were  delivered  more  promptly,  and  a  higher  fraction  of  moderately  and  very  sick 
patients  received  monitoring  with  ICU  or  telemetry.  Additional  data  about  performance  on 
process  criteria  and  scales  pre-  compared  with  post-PPS  are  shown  in  App.  C. 

DISCUSSION 

We  have  attempted  to  demonstrate  the  validity  of  our  process  measures  by  studying 
face  and  content  validity  and  by  establishing  process-outcome  links.  Validation  by  these 
methods  supports  the  view  that  our  explicit  process  criteria  are  indeed  measuring  important 
aspects  of  inhospital  processes  of  care.  If  our  process  scores  reflected  only  recording  changes 
rather  than  what  truly  happened  to  patients,  we  would  have  been  unable  to  find  a  statisti- 
cally significant  relationship  between  better  processes  of  care  and  lower  mortality.  In  addi- 
tion, if  we  were  measuring  only  improvements  in  recording  post-PPS  compared  with  pre- 
PPS,  we  would  have  expected  to  find  improvements  in  process  post-PPS  only  for  process 
measures  that  depend  heavily  on  recording  (e.g.,  physician  and  nurse  process).  However,  we 
have  demonstrated  a  significant  process-outcome  relationship  and  improvements  over  time 
in  all  process  scales,  including  both  the  recording-sensitive  scales  and  those  not  likely  to  be 
sensitive  to  changes  in  recording  practices. 

Having  developed  a  set  of  valid  process  measures,  we  compared  processes  of  care  pre- 
and  post-PPS.  Consistently  across  diseases  and  across  types  of  process  measures,  we  found 
processes  to  be  better  during  the  post-PPS  period.  Cognitive  and  technical  services  showed 
equivalent  gains.  The  findings  are  especially  notable  since  the  process  criteria  are  based 
upon  concepts  in  medicine  that  have  been  well  known  for  at  least  a  decade,  rather  than  upon 
new  and  evolving  technologies. 


Table  4.13 

PROCESS  CRITERIA  FOR  CONGESTIVE  HEART  FAILURE 


Af  Potionta 
/c  ui  i^allcIlLs 

%  of  Patients  Meeting 

to  wnom 

Process  Criteria 

Criteria 

Tf 
11 

i  nen 

Apply 

Pre-PPS 

Post-PPS 

All  patients 

Within  the  initial  two  days  of  hospitalization,  the 

hospitalized  for  CHF 

physician  should  document  each  of  the  following  in 

the  medical  record: 

Past  surgical  history 

100 

61 

66 

Medication  use  immediately  before  admission 

100 

78 

84 

Allergy  history 

100 

51 

58 

Alcohol  use  (or  nonuse) 

100 

42 

47 

Jugular  veins  examined 

100 

63 

70 

Heart  sounds  examined 

100 

67 

73 

Heart  murmur  examined  (or  not  present) 

100 

78 

80 

Abdomen  examined 

100 

97 

98 

Lower  extremity  examined  for  edema 

100 

91 

92 

Lung  examined  day  1 

100 

QQ 

QQ. 

Lung  examined  day  2 

^Q. 

71 

All  patients 

Within  the  initial  two  days  of  hospitalization,  the 

hospitalized  for  CHF 

nurse  should  document  each  of  the  following  in 

the  medical  record: 

Shortness  of  breath  (presence  or  absence)  day  1 

100 

88 

93 

Shortness  of  breath  (presence  or  absence)  day  2 

100 

QQ 

0,9 

DKJULi  pressure  assessment  ai  leasi  o  iimcs  uay  t. 

7Q 
/  O 

Q^l 
04 

Urinary  continence  (presence  or  absence) 

100 

DO 

79 

Ambulatory  status 

100 

7H 
i\3 

7Q 

Preadmission  residence 

100 

Q9 

y  u 

All  patients 

Within  the  initial  two  days  of  hospitalization,  order: 

hospitalized  for  CHF 

Chest  X-ray 

100 

94 

96 

i  ,  !  L  L  L  £  ULDl  UlUKi  dill 

100 

87 

91 

AJ1  patients 

Within  the  initial  two  days  of  hospitalization, 

hospitalized  for  CHI1 

order  each  of  the  following  laboratory  studies: 

BUN 

100 

88 

93 

Creatinine 

100 

69 

81 

Venous  bicarbonate 

100 

87 

utIUl  Villi 

100 

93 

97 

Potassium 

100 

93 

97 

White  blood  count 

100 

92 

94 

Hematocrit  or  hemoglobin 

100 

91 

94 

Patients  moderately 

Order  cardiac  enzyme  study  days  1  or  2 

26 

66 

67 

or  very  sick  (i.e., 

(e.g.,  CPK) 

ASCCa  z  5)  day  1  or  2 

Patients  moderately 

Obtain  arterial  blood  gas  days  1  or  2 

26 

72 

78 

or  very  sick  (i.e., 

ASSC  z  5)  day  1  or  2 

Patients  moderately 

Use  the  ICU  day  1  or 

16 

43 

46  ICU 

sick  (i.e.,  ASSC  5-6) 

Use  telemetry  day  1 

8 

24  Telemetry 

day  1 

Patients  moderately 

Use  the  ICU  day  2  or 

7 

49 

41  ICU 

sick  (i.e.,  ASSC  5-6) 

Use  telemetry  day  2 

13 

31  Telemetry 

day  2 
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Table  4.13 — continued 


%  of  Patients 
to  Whom 
Criteria 
Apply 

%  of  Patients  Meeting 
Process  Criteria 

If 

Then 

Pre-PPS 

Post-PPS 

Patients  very  sick 
(i.e.,  ASSC  a  7) 
day  1 

Use  the  ICU  day  1 

Telemetry  not  sufficient  but  preferable 
to  no  cardiac  monitoring  day  1 

6 

61 
12 

71  ICU 

17  Telemetry 

Patients  very  sick 
(i.e.,  ASSC  a  7) 
day  2 

Use  the  ICU  day  2 

Telemetry  not  sufficient  but 

preferable  to  no  cardiac  monitoring  day  2 

2 

60 
10 

69  ICU 

22  Telemetry 

Patients  hypoxic 
(i.e.,  p02  <  60)  on 
day  1 

Use  supplemental  oxygen  therapy  or 
intubate  day  1 

18 

87 

93 

Patients  hypoxic 
(i.e.,  p02  <  60)  on 
day  2 

Use  supplemental  oxygen  therapy  or 
intubate  day  2 

4 

90 

91 

Patients  moderately 
or  very  sick  (i.e., 
ASSC  a  5)  day  1  and 
day  2  and  arterial 
blood  gases  not 
evaluated  on  either 
day 

Use  supplemental  oxygen  therapy  or 
intubate  day  1  or  2 

1 

100 

86 

All  patients  hospitalized 
with  CHF  except  those 
with  systolic  blood 
pressure  <  90 
day  1  and  2 

Use  loop  diuretic  or  digitalis  day  1  or  2 

92 

94 

95 

All  nfltipntQ 

hospitalized  for  CHF 

On  hnQniffll  dnv  3  tnp  nnvsicinn  qnmilH 

document  each  of  the  following  in  the 
medical  record: 

Shortness  of  breath  (presence  or  absence) 
Lung  examination 

100 
100 

45 
60 

54 
69 

All  patients 
hosnitfiliypd  for*  C1HF 

On  hospital  day  3,  the  nurse  should 

document  in  the  medical  rpcnrd* 

Shortness  of  breath  (presence  or  absence) 

100 

85 

91 

Patients  moderately 
or  very  sick  (i.e.  ASSC 
a  5)  on  day  3 

Order  a  chest  X-ray  on  day  3 
Order  an  electrocardiogram  on  day  3 

7 
7 

41 

46 

29 
32 

Patients  somewhat 
sick  (i.e.,  ASSC 
3-4)  on  day  3 

Order  at  least  three  of  seven  venous  blood  studies 
on  day  3  (e.g.,  venous  bicarbonate,  creatinine,  BUN, 
sodium,  potassium,  white  blood  count,  hematocrit) 

16 

38 

54 

Patients  moderately 
or  very  sick  (i.e., 
ASSC  a  5)  on 
day  3 

Order  each  of  the  following  venous  blood  tests 
(e.g.,  venous  bicarbonate,  creatinine,  BUN,  sodium, 
potassium,  white  blood  count,  hematocrit) 

7 

14 

16 

Patients  moderately  or 
very  sick  (i.e., 
ASSC  i  5)  on  day  3 

Order  a  cardiac  enzyme  study  (e.g.,  CPK)  day  3 

7 

28 

18 

Patients  moderately  or 

Obtain  an  arterial  blood  gas  day  3 

7 

42 

39 

very  sick  (i.e., 
ASSC  a  5)  on  day  3 
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Table  4.13 — continued 


%  of  Patients 
to  Whom 
Criteria 
Apply 

%  of  Patients  Meeting 
Process  Criteria 

If 

Then 

Pre-PPS 

Post-PPS 

Patients  moderately 
sick  (i.e.,  AoaO 
5-6)  on  day  3 

Use  the  ICU  day  3  or 
use  telemetry  day  3 

5 

47 
13 

39  ICU 

26  Telemetry 

Patients  very  sick 
(i.e.,  ASSC  a  7) 
on  day  3 

Use  the  ICU  day  3;  telemetry  is  not 
sufficient  but  is  preferable  to  no 
cardiac  monitoring  day  3 

2 

82 

0 

61  ICU 

19  Telemetry 

Patients  hypoxic 
(i.e.,  p02  <  60) 
on  day  3 

Use  supplemental  oxygen  therapy  or  intubate  day  3 

3 

87 

98 

All  patients  hospitalized 

Use  loop  diuretic  or  digitalis  day  3 

94 

91 

92 

for  CHF  except  those 
with  systolic  BP  <  90 
day  3 

aAny  symptom  or  sign  complex  (ASSC)  is  defined  by  a  weighted  sum  of  scores  assigned  for  the  presence  of  each  of 
the  following  on  any  given  days.  Weights  are  assigned  as  follows:  shortness  of  breath,  1  point;  chest  pain,  1  point;  con- 
fusion, 2  points;  heart  rate  a  130  beats  per  minute,  2  points;  respiratory  rate  a  30  per  minute,  2  points;  diastolic  blood 
pressure  a  105  mm  Hg,  1  point;  systolic  blood  pressure  <  90  mm  Hg,  3  points. 
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Table  4.14 

PROCESS  CRITERIA  FOR  ACUTE  MYOCARDIAL  INFARCTION 

%  of  Patients        %  of  Patients  Meeting 
to  Whom              Process  Criteria 
Criteria   


If 

Then 

Atjrjlv 

Pre-PPS 

Post-PPS 

All  patients 

Within  the  initial  two  days  of  hospitalization,  the 

hospitalized 

physician  should  document  each  of  the  following  in 

for  AMI 

the  medical  record: 

1  ItVlUUS  La.IUlfci*_  I !  Io  LUl  J' 

100 

87 

Time  of  onset  of  AMI  symptoms 

100 

95 

96 

Social  or  family  history 

100 

64 

64 

Medication  use  immediately  before  admission 

100 

81 

86 

Allergy  history 

100 

56 

62 

Alcohol  use  (or  nonuse)  or  smoking  habits 

100 

61 

64 

ju  guitar  veins  examined 

ft! 

DI 

DO 

XlcfcU  t  sUUIlUs  cXallllllcU 

7fi 

0 J. 

Heart  murmurs  examined  (or  not  present) 

i  fin 

04 

04 

Abdomen  examined 

i  r\r\ 

1UU 

Q7 

07 

y  / 

LUVVcI    L  A  LI  Cill  1  \.y  CA  HI  IliJ  lcU  1UI  trLlLXJlcl 

100 

7ft 

R9 

Oil 

*oiiuai  pain  ^prtJociiCc  or  ausf  nct;j  uay  1 

l  fifi 

174 

Chest  pain  foresence  or  absence)  day  2 

100 

63 

71 

Lung  examination  day  1 

100 

96 

97 

Lung  examination  day  2 

100 

64 

70 

All  patients 

Within  the  initial  two  days  of  hospitalization, 

hospitalized 

the  nurse  should  document  each  of  the 

for  AMI 

following  in  the  medical  record: 

Chest  pain  (presence  or  absence)  day  1 

100 

92 

93 

Chest  pain  (presence  or  absence)  day  2 

100 

92 

92 

Premature  ventricular  contractions  assessed  day  1 

100 

63 

67 

Premature  ventricular  contractions  assessed  day  2 

100 

71 

75 

Weight  day  1  or  2 

100 

58 

65 

Cardiac  monitor  rhythm  strips  (at  least  3  on  day  1) 

100 

76 

86 

Preadmission  residence 

100 

92 

94 

All  patients 

Within  the  initial  two  days  of  hospitalization,  order 

hospitalized 

each  of  the  following: 

for  AMI 

Chest  X-ray  day  1  or  2 

100 

91 

93 

Electrocardiogram  day  1 

100 

94 

97 

Electrocardiogram  day  2 

100 

90 

91 

All  patients 

Within  the  initial  two  days  of  hospitalization, 

hospitalized 

order  each  of  the  following  laboratory  studies: 

for  AMI 

RTTN 

Dull 

i  no 

Rfl 
oo 

u  1 

Creatinine 

100 

66 

79 

Venous  bicarbonate 

100 

81 

88 

Sodium 

100 

92 

95 

Potassium 

100 

93 

95 

White  blood  count 

100 

94 

96 

Hematocrit  or  hemoglobin 

100 

95 

96 

All  patients 

Order  cardiac  enzyme  study  day  1 

100 

40 

49  CPK  or  LDH 

hospitalized 

(e.g.,  total  CPK  or  lactate  dehydrogenase 

isoenzymes 

for  AMI 

(LDH)  isoenzimes  or  CPK  isoenzymes) 

46 

45  Total  CPK  only 

All  patients 

Order  cardiac  enzyme  study  day  2 

100 

51 

56  CPK  or  LDH 

hospitalized 

(e.g.,  total  CPK  or  lactate  dehydrogenase 

isoenzymes 

for  AMI 

(LDH)  isoenzymes  or  CPK  isoenzyme) 

37 

34  Total  CPK  only 

61 


Table  4.14 — continued 


%  of  Patients 


of  Patients  Meeting 


If 


Then 


to  Whom 

Criteria   

Apply  Pre-PPS 


Process  Criteria 


Post-PPS 


Patients  moderately 
or  very  sick  (i.e., 
ASSC  a  5)  day  1  or  2 

Patients  with  cardiogenic 
shock  with  end  organ 
damage,  arrhythmias,  and 
very  sick  (i.e.,  ASSC 
2  7)  day  1 

Patients  with  shock 
or  very  sick  (i.e., 
ASSC  2  7)  day  1 

Patients  moderately  sick 
(i.e.,  ASSC  =  5-6)  day  1 

Patients  with  congestive 
heart  failure  and  somewhat 
sick  (i.e.,  ASSC  =  3-4) 
day  1 

Patients  with  congestive 
heart  failure  and 
not  very  sick  (i.e., 
ASSC  s  2)  day  1 

Patients  somewhat  sick 
(i.e.,  ASSC  =  3-4)  day  1 

Patients  not  very  sick 
but  some  symptom  or  sign 
(i.e.,  ASSC  =  2)  day  1 

Patients  with  AMI  and  one 
or  no  symptoms  or  signs 
(i.e.,  ASSC  =  0-1) 
day  1 

Patients  with  no  evidence 
of  AMI  and  one  or  no 
symptoms  or  signs  (i.e., 
ASSC  =  0-1)  day  1 

Patients  with  cardiogenic 
shock  with  end  organ 
damage,  arrhythmias,  and 
very  sick  (i.e.,  ASSC 
a  7)  day  2 

Patients  with  shock  or 
very  sick  (i.e.,  ASSC 
a  7)  day  2 

Patients  moderately  sick 
(i.e.,  ASSC  =  5-6)  day  2 

Patients  with  congestive 
heart  failure  and  somewhat 
sick  (i.e.,  ASSC  =  3-4) 
day  2 


Obtain  arterial  blood  gas  day  1  or  2 


Use  ICU  day  1;  telemetry  is  not  sufficient  but 
is  preferable  to  no  cardiac  monitoring 


Use  ICU  day  1;  telemetry  is  not  sufficient  but 
is  preferable  to  no  cardiac  monitoring 

Use  ICU  day  1;  telemetry  is  not  sufficient  but 
is  preferable  to  no  cardiac  monitoring 

Use  ICU  day  1;  telemetry  is  not  sufficient  but 
is  preferable  to  no  cardiac  monitoring 


Use  ICU  day  1;  telemetry  is  not  sufficient  but 
is  preferable  to  no  cardiac  monitoring 


Use  ICU  day  1;  telemetry  is  not  sufficient  but 
is  preferable  to  no  cardiac  monitoring 

Use  ICU  day  1  or  use  telemetry  day  1;  ICU  is 
preferable 

Use  ICU  day  1  or  use  telemetry  day  1;  ICU  is 
preferable 


Use  ICU  day  1  or  use  telemetry  day  1 


Use  ICU  day  2;  telemetry  is  not  sufficient  but 
is  preferable  to  no  cardiac  monitoring 


Use  ICU  day  2;  telemetry  is  not  sufficient  but 
is  preferable  to  no  cardiac  monitoring 

Use  ICU  day  2;  telemetry  is  not  sufficient  but 
is  preferable  to  no  cardiac  monitoring 

Use  ICU  day  2;  telemetry  is  not  sufficient  but 
is  preferable  to  no  cardiac  monitoring 


34 


16 

8 
8 

11 

14 
19 

15 


13 

4 
3 


67 


95 
1 


88 
8 

83 
9 

80 
10 


77 
14 


77 
11 

81 
11 

71 
20 


55 
23 


95 
1 


92 
5 

89 
11 

80 
12 


74 


91 
7 


89  ICU 
7  Telemetry 

82  ICU 

15  Telemetry 

80  ICU 

14  Telemetry 


75  ICU 

19  Telemetry 


75  ICU 

22  Telemetry 

70  ICU 

26  Telemetry 

77  ICU 

20  Telemetry 


71  ICU 

29  Telemetry 


91  ICU 
7  Telemetry 


87  ICU 

10  Telemetry 

85  ICU 
9  Telemetry 

93  ICU 
5  Telemetry 


62 


Table  4.14 — continued 

%  of  Patients 
to  Whom 
Criteria 
Apply 

%  of  Patients  Meeting 
Process  Criteria 

if 

Then 

Pre-PPS 

Post-PPS 

Patients  with  congestive 
heart  failure  and 
not  very  sick  (i.e., 
ASSC  s  2)  day  2 

Use  ICU  day  2;  telemetry  is  not  sufficient  but 
is  preferable  to  no  cardiac  monitoring 

7 

91 
5 

89  ICU 

10  Telemetry 

j 

Patients  somewhat  sick 
(i.e.,  ASSC  =  3-4)  day  2 

Use  ICU  day  2;  telemetry  is  not  sufficient  but 
is  preferable  to  no  cardiac  monitoring 

10 

86 
7 

82  ICU 

15  Telemetry 

Patients  at  least  somewhat 
sick  (i.e.,  ASSC  a  3)  on 

Use  ICU  day  2;  telemetry  is  not  sufficient  but 
is  preferable  to  no  cardiac  monitoring 

29 

77 
14 

78  ICU 

18  Telemetry 

day  1  with  or  without 
shock  and  congestive 
heart  failure  but  not 
very  sick  (i.e.,  ASSC 
s  2)  on  day  2  with  or 
without  evidence  of  AMI 

Patients  not  very  sick 
(i.e.,  ASSC  s  2)  on  day  1 
with  or  without  evidence 
of  AMI  and  not  very  sick 
but  some  symptom  or  sign 
(i.e.,  ASSC  =  2)  day  2 

Patients  not  very  sick 
(i.e.,  ASSC  s  2)  on  day  1 
and  with  AMI  and  one  or 
no  symptoms  or  signs 
(i.e.,  ASSC  =  0-1) 
day  2 

Patients  not  very  sick 
(i.e.,  ASSC  s  2)  on  day  1 
and  with  no  evidence  of 
AMI  and  one  or  no  symptoms 
or  signs  (i.e.,  ASSC  =  0-1) 
day  2 

Patients  with  cardiogenic 
shock  with  end  organ 
damage,  arrhythmias,  and 
very  sick  (i.e.,  ASSC  a  7) 
day  1  or  2 

Patients  with  cardiogenic 
shock  with  end  organ 
damage,  arrhythmias,  and 
very  sick  (i.e.,  ASSC  a  7) 
day  1 

Patients  with  cardiogenic 
shock  with  end  organ 
damage,  arrhythmias,  and 
very  sick  (i.e.,  ASSC  *  7) 
day  2 


Use  ICU  day  2  or  use  telemetry  day  2;  ICU 
is  preferable 


Use  ICU  day  2  or  use  telemetry  day  2;  ICU 
is  preferable 


Monitoring  day  2  in  ICU  or  telemetry  is  preferable 
to  no  monitoring 


Place  a  Swan  Ganz  catheter  on  day  1  or  2 


Use  pressors  or  IV  unloading  agents  day  1 


Use  pressors  or  IV  unloading  agents  day  2 


22 


38       78  ICU 
4       22  Telemetry 


76       75  ICU 

17       24  Telemetry 


65       67  ICU 

20       32  Telemetry 


31  38 


71  76 


65  85 


63 


Table  4.14 — continued 


if 


Then 


%  of  Patients        %  of  Patients  Meeting 
to  Whom  Process  Criteria 

Criteria   

Apply  Pre-PPS  Post-PPS 


Patients  with  ventricular 
tachycardia  (VTach)  day  1 

Patients  with  PVCs  day  1 

Patients  with  VTach  day  2 

Patients  with  VTach  day  2 

Patients  with  3rd  degree 
heart  block  day  1  or  2 
and  anterior  MI 

Patients  with  3rd  degree 
heart  block  day  1  or  2 
and  no  anterior  MI 

Patients  given 
streptokinase 

Patients  given 
streptokinase 

Patients  with  CHF  but  no 
hypotension  day  1 

Patients  with  CHF  but  no 
hypotension  day  2 

Patients  with  hypertension 
(diastolic  BP  a  105) 
persisting  for  day  1  and  2 

Patients  with  chest  pain 
and  no  rV  unloading  agents 
on  day  2 

Patients  with  chest  pain 
day  1  and  2  and  no 
evidence  of  hypotension 
or  CHF  on  day  1  or  2  and 
no  IV  unloading  agents  on 
day  2 

Patients  with  chest  pain 
day  1  and  2  and  evidence 
of  hypotension  or  CHF 
on  day  1  or  2  and  no  IV 
unloading  agents  on  day  2 

All  patients 
hospitalized 
for  AMI 


All  patients 
hospitalized 
for  AMI 


Give  antiarrhythmics  day  1 

Give  antiarrhythmics  day  1 
Give  antiarrhythmics  day  2 
Give  antiarrhythmics  day  2 
Use  pacemaker  day  1  or  2 

Use  pacemaker  day  1  or  2 

Document  bleeding  history  (presence  or  absence) 

Start  streptokinase  within  6  hours  of  the  time 
symptoms  were  first  noted 

Give  loop  diuretics  day  1 

Give  loop  diuretics  day  2 

Give  rV  unloading  agents,  long-acting  nitrates, 
oral  or  FV  beta  blockers,  or  calcium  channel 
blockers  day  2 

Give  short-acting  nitrates  day  2 


Give  long-acting  nitrates,  oral  or  IV  beta  blockers, 
or  calcium  channel  blockers  on  day  2 


Give  long-acting  nitrates  or  calcium  channel 
blockers  on  day  2 


On  day  3  of  the  hospitalization,  the  physician 
should  document  each  of  the  following 
in  the  medical  record: 

Chest  pain  (presence  or  absence) 
Lung  examination 

On  day  3  of  the  hospitalization,  the  nurse  should 
document  each  of  the  following  in  the 
medical  record: 


1 

23 
1 

17 
1 


100  88 


2 
2 
19 
9 

3 

36 
19 


15 


69 
62 
62 
92 


69 
80 
65 
62 


44  78 

0  14 

40  72 

51  54 

63  70 

56  88 

54  54 

67  81 


65  78 


100  60  70 

100  64  70 


Chest  pain  (presence  or  absence) 
PVCs  assessed 


100 
100 


91  92 
69  71 


64 


Table  4.14 — continued 


%  of  Patients 
to  Whom 
Criteria 
Apply 

%  of  Patients  Meeting 
Process  Criteria 

If 

Then 

Pre-PPS 

Post-PPS 

Patients  moderately  or 
very  sick  (i.e.,  ASSC  *  5) 
on  day  3 

Order  chest  X-ray  on  day  3 

12 

51 

59 

Patients  with  chest  pain 
or  somewhat,  moderately, 
or  very  sick 
(i.e.,  ASSC  ^  3)  on 
day  3 

Order  electrocardiogram  on  day  3 

45 

81 

78 

Patients  somewhat  sick 
(i.e.,  ASSC  =  3-4)  on 
day  3 

Order  at  least  1  of  the  following  laboratory 
studies  on  day  3:  BUN,  creatinine,  sodium, 
potassium 

17 

54 

52 

Patients  moderately  or 
studies  very  sick 
on  day  3 

Order  at  least  2  of  the  following  laboratory 
on  day  3:  BUN,  creatinine,  sodium,  potassium 
(i.e.,  ASSC  a  5) 

12 

52 

71 

Patients  with  chest 
pain  day  3 

Order  cardiac  enzyme  study  day  3 
(e.g.,  creatinine  phosphokinase  (CPK)  or 
lactate  dehydrogenase  (LDH)  isoenzymes 
or  total  CPK) 

19 

38 

32 

Patients  moderately 
or  very  sick  (i.e., 
ASSC  a  5 ) day  3 

Obtain  arterial  blood  gas  day  3 

11 

42 

50 

Patients  very  sick 
(i.e.,  ASSC  z  7)  day  3 

Use  ICU  day  3;  telemetry 

is  not  sufficient  but  is  preferable  to  no 

(.uiuiut  muniiuring 

5 

94 
4 

94  ICU 
3  Telemetry 

Patients  moderately  sick 
but  is  (i.e.,  ASSC  =  5-6)  day  3 

Use  ICU  day  3;  telemetry  is  not  sufficient 
preferable  to  no  cardiac  monitoring 

7 

90 
10 

89  ICU 
8  Telemetry 

Patients  somewhat  sick 

but  is  (i.e.,  ASSC  =  3-4)  day  3 

Use  ICU  day  3;  telemetry  is  not  sufficient 
preferable  to  no  cardiac  monitoring 

17 

87 
9 

80  ICU 

16  Telemetry 

Patients  at  least  somewhat 
but  is  sick  (i.e.,  ASSC  a  3)  on 

Use  ICU  day  3;  telemetry  is  not  sufficient 
preferable  to  no  cardiac  monitoring 

19 

84 
9 

85  ICU 

13  Telemetry 

day  2  with  or  without 
shock  and  CHF  but  not 
very  sick  (i.e.,  ASSC 
s  2)  on  day  3  with  or 
without  evidence  of  AMI 

Patients  not  very  sick 
(i.e.,  ASSC  s  2)  on  day  2 
and  with  AMI  and  not  very 
sick  (i.e.,  ASSC  s  2)  day  3 

Patients  not  very  sick 
(i.e.,  ASSC  s  2)  on  day  2 
and  with  no  evidence  of 
AMI  and  not  very  sick 
(i.e.,  ASSC  s  2)  on  day  3 

Patients  with  hypotension 
and  moderately  or  very 
sick  (i.e.,  ASSC  *  5) 
day  3 


Use  ICU  day  3  or  use  telemetry  day  3;  ICU  is 
preferable 


Monitoring  day  3  in  ICU  or  telemetry  is  not 
necessary 


Use  pressors  or  rv  unloading  agents  day  3 


10 


77 
15 


62 
21 


52 


73  ICU 

24  Telemetry 


59  ICU 

35  Telemetry 


61 


65 


Table  4.14 — continued 


%  of  Patients 
to  Whom 

Apply 

%  of  Patients  Meeting 
Process  Criteria 

If 

Then 

Pre-PPS 

Post-PPS 

Patients  with  VTach 
day  1  or  2  or  PVCs  day  3 

Give  antiarrhythmics  or  oral  or  IVs  blockers  day  3 

10 

68 

63 

Patients  with  hypertension 
(diastolic  BP  a  105) 
persisting  for  day  2  and  3 

Give  IV  unloading  agents,  long-acting  nitrates, 
oral  or  IV  beta  blockers,  or  calcium  channel 
blockers  day  3 

1 

78 

88 

Patients  with  chest  pain 
and  no  IV  unloading  agents 
on  day  3 

Give  short-acting  nitrates  day  3 

26 

59 

47 

Patients  with  chest  pain 
day  2  and  3  and  no 
evidence  of  hypotension 
or  CHF  on  day  1  or  2  and 
no  IV  unloading  agents  on 
day  3 

Give  long-acting  nitrates,  oral  or  IV  beta  blockers, 
or  calcium  channel  blockers  on  day  3 

10 

73 

91 

Patients  with  chest  pain 
day  2  and  3  and  evidence 
of  hypotension  or  CHF 
on  day  1  or  2  and  no  IV 
unloading  agents  on  day  3 

Give  long-acting  nitrates  or  calcium  channel 
blockers  on  day  3 

8 

71 

84 

Any  symptom  or  sign  complex  (ASSC)  is  defined  by  a  weighted  sum  of  scores  assigned  for  the  presence  of  each  of  the 
following  on  any  given  days.  Weights  are  assigned  as  follows:  shortness  of  breath,  1  point;  chest  pain,  1  point;  confusion,  2 
points;  heart  rate  a  130,  2  points;  respiratory  rate  a  30  per  minute,  2  points;  diastolic  blood  pressure  a  105  mm  Hg,  1  point; 
systolic  blood  pressure  <  90  mm  Hg,  3  points. 


66 


Table  4.15 
PROCESS  CRITERIA  FOR  PNEUMONIA 


If 


Then 


%  of  Patients 
to  Whom 
Criteria 
Apply 


i  of  Patients  Meeting 
Process  Criteria 


Pre-PPS 


Post-PPS 


All  patients 
hospitalized  for  PNE 


All  patients 
hospitalized  for  PNE 


Patients  moderately  or 
very  sick  (i.e.,  ASSCa  2  5) 
day  1  or  2 

All  patients 
hospitalized  for  PNE 


Patients  moderately 
or  very  sick  (i.e., 
ASSC  z  5)  day  1  or  2 

Patients  moderately 
or  very  sick  (i.e., 
ASSC  a  5)  day  1  or  2 


Within  the  initial  two  days  of  hospitalization,  the 
physician  should  document  each  of  the  following  in 
the  medical  record: 

Sputum  or  cough  (presence  or  absence) 

Previous  pulmonary  disease  (presence  or  absence) 

Medication  use  immediately  before  admission 

Alcohol  use  (or  nonuse) 

Tobacco  use  (or  nonuse) 

Temperature 

Jugular  veins  examined 

Heart  murmurs  examined 

Lower  extremity  examined  for  edema 

Mental  status  assessed 

Lung  examination  day  1 

Lung  examination  day  2 

Shortness  of  breath  (presence  or  absence)  day  1 
Shortness  of  breath  (presence  or  absence)  day  2 

Within  the  initial  two  days  of  hospitalization,  the 
nurse  should  document  each  of  the  following  in 
the  medical  record: 

Shortness  of  breath  (presence  or  absence)  day  1 
Shortness  of  breath  (presence  or  absence)  day  2 
Blood  pressure  assessment  at  least  3  times  day  2 
Urinary  continence  (presence  or  absence) 
Ambulatory  status 
Preadmission  residence 

Within  the  initial  two  days  of  hospitalization,  order: 

Chest  X-ray 
Electrocardiogram 

Within  the  initial  two  days  of  hospitalization, 
order  each  of  the  following  laboratory  studies: 

BUN 

Creatinine 

Venous  bicarbonate 

Sodium 

Potassium 

White  blood  count 

Hematocrit  or  hemoglobin 

White  blood  cell  count  differential 

Order  cardiac  enzyme  study  day  1  or  2 
(e.g.,  CPK) 

Obtain  arterial  blood  gas  day  1  or  2 


100 
100 
100 
100 
100 
100 
100 
100 
100 
100 
100 
100 
100 
100 


100 
100 
100 
100 
100 
100 


31 
31 


100 
100 
100 
100 
100 
100 
100 
100 

31 


30 


79 
58 
71 
40 
47 
76 
41 
76 
68 
61 
98 
59 
66 
25 


67 
72 
69 
66 
69 
96 


99 
77 


83 
65 
77 
86 
87 
99 
98 
97 

35 


70 


77 
63 
75 
44 
52 
79 
48 
76 
75 
65 
98 
68 
74 
31 


78 
84 
79 
76 
78 
98 


100 
75 


90 
79 
84 
93 
93 
98 
98 
93 

34 


80 


67 


Table  4.15 — continued 


%  of  PatipnN 
to  Whom 
Criteria 
Apply 

%  of  Patients  Meeting 
Process  Criteria 

If 

Then 

Pre-PPS 

Post-PPS 

Patients  somewhat  sick 
(i.e.,  ASSC  =  3-4) 
day  1 

Use  the  ICU  day  1  or 
use  telemetry  day  1 

29 

7 
4 

9  ICU 
10  Telemetry 

Patients  somewhat  sick 
(i.e.,  ASSC  =  3-4) 

j  n 

day  A 

Use  the  ICU  day  2  or 
use  telemetry  day  2 

17 

19 
5 

17  ICU 

11  Telemetry 

Patients  moderately 
sick  (i.e.,  ASSC  5-6) 
day  1 

Use  the  ICU  day  1  or 
use  telemetry  day  1 

18 

19 
7 

18  ICU 

13  Telemetry 

Patients  moderately 
sick  (i.e.,  ASSC  5-6) 
day  i 

Use  the  ICU  day  2  or 
use  telemetry  day  2 

8 

31 
4 

35  ICU 

12  Telemetry 

Patients  very  sick 
(i.e.,  ASSC  a  7) 

Ami  1 

day  1 

Use  the  ICU  day  1; 

telemetry  is  not  sufficient  but  is 

preferable  to  no  cardiac  monitoring  day  1 

10 

42 

5 

50  ICU 
7  Telemetry 

Patients  very  sick 
(i.e.,  ASSC  a  7) 
day  I 

Use  the  ICU  day  2; 

telemetry  is  not  sufficient  but  is 

preferable  to  no  cardiac  monitoring  day  2 

4 

52 
11 

53  ICU 

17  Telemetry 

Patients  hypoxic 
(i.e.,  p02  <  60)  on 
day  1 

Use  supplemental  oxygen  therapy  or 
intubate  day  1 

27 

83 

90 

Patients  hypoxic 
(i.e.,  p02  <  60)  on 
day  2 

Use  supplemental  oxygen  therapy  or 
intubate  day  2 

8 

94 

93 

Patients  moderately 
or  very  sick  (i.e., 
ASSC  a  5)  day  1  and 
day  2  and  arterial 
blood  gases  were  not 
evaluated  on  either  day 

Use  supplemental  oxygen  therapy  or 
intubate  day  1  or  2 

1 

77 

64 

All  nonimmunocompromised 
with  PNE 

Use  antibiotics  within  the  first  4  hours 

on 

yu 

28 

32 

All  immunocompromised 
patients  hospitalized 
with  PNE 

Use  antibiotics  within  the  first  2  hours 

i  n 
1U 

3 

4 

Patients  with  fever 
(T>  101)  on  day  lor  2 

Order  blood  cultures  day  1  or  2 

42 

56 

68 

Patients  with  fever 
(T  >  101)  on  day  1  or  2 

Order  urine  culture  or  urinalyses  day  1  or  2 

42 

88 

82 

Patients  with  sputum  for 
culture  day  1  or  2 

Document  sputum  gram  stain  quality  day  1  or  2 

53 

59 

68 

All  patients 
hospitalized  for  PNE 

On  hospital  day  3,  the  physician  should 
document  each  of  the  following  in  the 
medical  record: 

Shortness  of  breath  (presence  or  absence) 
Lung  examination 

100 
100 

21 
59 

27 
65 

68 


Table  4.15 — continued 


%  of  Patients 
to  Whom 
Criteria 
Apply 

%  of  Patients  Meeting 
Process  Criteria 

If 

Then 

Pre-PPS 

Post-PPS 

All  patients 
hospitalized  for  PNE 

On  hospital  day  3,  the  nurse  should 
document  in  the  medical  record: 

Shortness  of  breath  (presence  or  absence) 

100 

72 

83 

Patients  moderately 
or  very  sick  (i.e.,  ASSC 
a  5)  on  day  3 

Order  a  chest  X-ray  or  an  electrocardiogram  on  day  3 

9 

51 

56 

Patients  somewhat  sick 
(i.e.,  ASSC  3-4)  on 
day  3 

Order  at  least  1  of  4  venous  blood  studies  on  day  3 
(e.g.,  creatinine,  BUN,  sodium,  potassium) 

13 

43 

42 

Patients  moderately  or 
very  sick  (i.e.,  ASSC 
a  5)  on  day  3 

Order  at  least  2  of  4  venous  blood  tests 
(e.g.,  creatinine,  BUN,  sodium,  potassium) 

9 

55 

55 

Patients  moderately  or 
very  sick  (i.e.,  ASSC 
a  5)  on  day  3 

Order  a  cardiac  enzyme  study  (e.g.,  CPK)  day  3 

9 

16 

16 

Patients  moderately  or 
very  sick  (i.e.,  ASSC 
a  5)  on  day  3 

Obtain  an  arterial  blood  gas  day  3 

9 

48 

59 

Patients  somewhat  sick 
(i.e.,  ASSC  3-4)  on  day  3 

Use  the  ICU  day  3  or 
use  telemetry  day  3 

13 

22 
7 

16  ICU 

11  Telemetry 

Patients  moderately  sick 
(i.e.,  ASSC  5-6)  on  day  3 

Use  the  ICU  day  3  or 
use  telemetry  day  3 

6 

30 
9 

36  ICU 
9  Telemetry 

Patients  very  sick 
(i.e.,  ASSC  a  7)  on  day  3 

Use  the  ICU  day  3; 

telemetry  is  not  sufficient  but  is 

preferable  to  no  cardiac  monitoring  day  3 

3 

55 
17 

72  ICU 
8  Telemetry 

Patients  hypoxic 

(i.e.,  p02  <  60)  on  day  3 

Use  supplemental  oxygen  therapy  or  intubate  day  3 

5 

93 

90 

Any  symptom  or  sign  complex  (ASSC)  is  defined  by  a  weighted  sum  of  scores  assigned  for  the  presence  of  each  of  the  fol- 
lowing on  any  given  days.  Weights  are  assigned  as  follows:  shortness  of  breath,  1  point;  chest  pain,  1  point;  confusion,  2 
points;  heart  rate  a  130  beats  per  minute,  2  points;  respiratory  rate  a  30  per  minute,  2  points;  diastolic  blood  pressure  a  105 
mm  Hg,  1  point;  systolic  blood  pressure  <  90  mm  Hg,  3  points. 
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Table  4.16 

PROCESS  CRITERIA  FOR  CEREBROVASCULAR  ACCIDENT 


%  of  Patients 

%  of  Patients  Meeting 

to  Whom 

Process  Criteria 

Criteria 

If 

Then 

Apply 

Pre-PPS 

Post-PPS 

All  nntipnt^ 

Jul    M  U  H  V 1 1  k-u 

Within  the  initial  two  days  of  hospitalization,  the 

physician  should  document  each  of  the  following  in 

for  CVA 

the  medical  record: 

Previous  cardiac  history  or  hypertension 

100 

81 

82 

Time  of  onset  of  neurologic  symptoms 

100 

91 

94 

Prior  CVA/TIA 

100 

48 

53 

Medication  use  immediately  before  admission 

100 

74 

82 

Pupils  assessed 

100 

80 

82 

Gag  reflex  assessed 

100 

35 

38 

V/alULIU  Ul  UH3  aSSCDOCU 

100 

01 

1-4  a  a  ft  sniinnQ  nQqpQCpii 

XlCal  t  5IJUI1UB  OBOC5JCU 

100 

57 

63 

Muscle  tone/strength  assessed 

100 

82 

86 

Limb  reflexes  assessed 

100 

64 

71 

Mental  status  assessed 

100 

83 

83 

Neurologic  deficits  (presence  or  absence)  day  1 

100 

99 

99 

Neurologic  deficits  (presence  or  absence)  day  2 

100 

74 

85 

Visit  to  patient  day  1 

100 

100 

100 

Visit  to  patient  day  2 

100 

92 

96 

All  patients 

Within  the  initial  two  days  of  hospitalization,  the 

hospitalized 

nurse  should  document  each  of  the  following  in 

for  CVA 

the  medical  record: 

Nutritional  intake  assessed  day  1 

100 

63 

62 

Nutritional  intake  assessed  day  2 

100 

88 

90 

Pupils  assessed  day  1 

100 

43 

62 

Pupils  assessed  day  2 

1UU 

38 

57 

Blood  pressure  measured  at  least  3  times  on  day  2 

i  r\n 
1UU 

79 

86 

All  patients 

Within  the  initial  two  days  of  hospitalization,  either 

hospitalized 

the  physician  or  the  nurse  should  document  each  of  the 

for  CVA 

following  in  the  medical  record: 

Urinary  continence  (presence  or  absence) 

1UU 

60 

68 

Ambulatory  status 

1UU 

73 

80 

Preadmission  residence 

100 

97 

97 

All  patients 

Within  the  initial  two  days  of  hospitalization,  order 

hospitalized 

each  of  the  following: 

tor  OVA 

Chest  X-ray 

100 

80 

79 

Electrocardiogram 

100 

82 

86 

All  patients 

\Vlthin  thp  initial  twn  HnVQ  nf  Vin<anitflli7fltinn  nrHpr 

hospitalized 

each  of  the  following  laboratory  studies: 

for  CVA 

BUN 

100 

84 

92 

Creatinine 

100 

62 

79 

Venous  bicarbonate 

100 

78 

86 

Sodium 

100 

88 

94 

Potassium 

100 

88 

94 

White  blood  count 

100 

94 

95 

Hematocrit  or  hemoglobin 

100 

94 

95 

70 


Table  4.16 — continued 


%  of  Patients 
to  Whom 
Criteria 
Apply 

%  of  Patients  Meeting 
Process  Criteria 

If 

Then 

Pre-PPS 

Post-PPS 

Patients  moderately, 
or  very  sick  (i.e., 
Aoou  i  1)  day  1  or  & 

Obtain  arterial  blood  gas  day  1  or  2 

27 

47 

57 

Patients  moderately  sick 

~    a  cor1     o  o^  1 

(i.e.,  Aoso  =  4-o)  day  1 

Use  ICU  day  1 

14 

30 

37 

Patients  very  sick 

(i  n      AGO/*1  1 

u.e.,  AotsU  a4j  day  1 

Use  ICU  day  1 

6 

54 

58 

Patients  moderately  sick 
(i.e.,  ASSC  =  2-3)  day  2 

Use  ICU  day  2 

12 

35 

39 

Patients  very  sick 
(i.e.,  ASSC  a  4)  day  2 

Use  ICU  day  2 

4 

48 

54 

Posturing,  no  response 
to  pain,  or  coma 

Do  CT  scan,  carotid  angiography,  brain  scan,  encephalogram, 
or  lumbar  puncture  day  1  or  2 

26 

34 

60c 

All  patients 
hospitalized 
for  CVA 

On  day  3  of  the  hospitalization,  the  physician  should  document 
each  of  the  following  in  the  medical  record: 

Neurologic  deficits  (presence  or  absence) 
Lung  examination 
Visit  to  patient 

100 
100 
100 

68 
29 
91 

78 
41 
96 

All  patients 
hospitalized 
for  CVA 

On  day  3  of  the  hospitalization,  the  nurse  should  document 
each  of  the  following  in  the  medical  record: 

Nutritional  intake  assessed 
Pupils  assessed 

100 
100 

88 
31 

90 
49 

Patients  moderately  sick 
(i.e.,  ASSC  =  2-3)  on 
day  3 

Order  at  least  3  of  the  following  laboratory  studies  on  day  3: 
BUN,  creatinine,  sodium,  potassium,  white  blood  cell  count, 
hematocrit  or  hemoglobin,  venous  bicarbonate 

10 

47 

38 

Patients  very  sick 
(i.e.,  ASSC  z  4) 
on  day  3 

Order  at  least  4  of  the  following  laboratory  studies  on  day  3: 
BUN,  creatinine,  sodium,  potassium,  white  blood  cell  count, 
hematocrit  or  hemoglobin,  venous  bicarbonate 

3 

50 

51 

Patients  moderately  sick 
(i.e.,  ASSC  =  2-3)  day  3 

Use  ICU  day  3 

10 

34 

36 

Patients  very 

Use  ICU  day  3 

3 

58 

56 

sick  (i.e.,  ASSC  a  4) 
day  3 


Any  symptom  or  sign  complex  (ASSC)  is  denned  by  a  weighted  sum  of  scores  assigned  for  the  presence  of  each  of  the  follow- 
ing on  any  given  days.  Weights  are  assigned  as  follows:  shortness  of  breath,  1  point;  heart  rate  a  130  beats  per  minute,  2  points; 
respiratory  rate  *  30  per  minute,  2  points;  diastolic  blood  pressure  a  105  mm  Hg,  1  point;  systolic  blood  pressure  <  90  mm  Hg,  3 

points. 


Table  4.17 
PROCESS  CRITERIA  FOR  HIP 


%  of  Patients   %  of  Patients  Meeting 
to  Whom  Process  Criteria 

Criteria 


If 

Then 

Apply 

Pre-PPS 

Post-PPS 

All        _  j_  ■  j  

All  patients 

Within  the  initial  two  days  of  hospitalization,  the 

ho^ni£flli7ed 

physician  should  document  each  of  the  following  in 

for  HIP 

the  medical  record: 

Past  surgical  history 

1  fin 

J.UU 

( 0 

77 

Medication  use  immediatelv  before  admission 

100 

98 

98 

Prior  living  arrangements 

100 

70 

73 

Alcohol  use  (or  nonuse)  or  smoking  habits 

100 

44 

52 

Circumstances  of  hip  injury 

100 

98 

98 

Neck  examined 

100 

90 

90 

Heart  examined 

100 

98 

98 

Lungs  examined 

100 

99 

98 

Lower  extremity  examined  for  pulses 

100 

62 

67 

Mental  status  assessed 

i  on 

Do 

7n 

All  patients 

Within  the  initial  two  days  of  hospitalization,  the 

hospitalized 

nurse  should  document  each  of  the  following  in 

for  HIP 

the  medical  record: 

Urinary  continence  (presence  or  absence) 

100 

62 

69 

Ambulatory  status 

100 

85 

92 

Preadmission  residence 

100 

97 

98 

All  patients 

Within  the  initial  two  days  of  hospitalization,  order: 

hospitalized 

for  HIP 

Chest  X-ray 

100 

92 

93 

Electrocardiogram 

100 

90 

93 

All  patients 

Within  the  initial  two  days  of  hospitalization, 

hospitalized 

order  each  of  the  following  laboratory  studies: 

for  HIP 

BUN 

100 

85 

87 

Creatinine 

100 

59 

74 

Sodium 

100 

88 

92 

Potassium 

100 

89 

94 

White  blood  count 

100 

96 

97 

Hematocrit  or  hemoglobin 

100 

96 

97 
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Using  our  set  of  validated  process  criteria,  we  have  an  excellent  starting  point  for  con- 
tinuing to  understand  the  strengths  and  weaknesses  of  care  provided  to  hospitalized  Medi- 
care patients  in  the  United  States.  The  data  suggest  that  there  is  room  for  improvement  in 
most  aspects  of  care.  The  strong  relationships  between  physician  and  nurse  cognitive  mea- 
sures and  mortality  confirm  the  importance  of  clinicians  as  gatherers  of  data.  We  hope  that 
physicians  and  nurses  will  use  our  results  as  encouragement  to  take  the  time  necessary  to 
gather  and  record  essential  data  elements,  and  that  the  payment  structure  for  clinicians  will 
acknowledge  that  physicians'  cognitive  services  have  an  important  effect  on  outcomes.  The 
findings  of  better  outcomes  for  patients  with  better  technical  diagnostic  and  therapeutic 
processes  encourage  us  that  the  technical  advances  of  modern  medicine  do,  in  fact,  also  make 
a  difference.  Recognizing  these  important  relationships,  efforts  should  be  made  to  identify 
the  types  of  patients,  physicians,  and  hospitals  falling  short  in  the  performance  of  process. 
We  should  then  identify  reasons  for  the  shortfalls  and  correct  the  problems.  Our  data 
demonstrate  that  lives  are  at  stake. 

The  lack  of  a  consistent  process-outcome  relationship  for  scales  based  on  ICU  and 
telemetry  monitoring  is  disappointing.  We  believe  that  the  problem  lies  in  our  imperfect 
measurement  of  the  if  in  the  if-then  process  statements.  Ideally,  we  would  like  to  demon- 
strate improved  outcomes  if  monitoring  with  ICU  and  telemetry  is  appropriate  and  if  the 
patient  has  a  treatable  disease.  More  work  is  needed  to  better  understand  how  to  identify 
the  group  of  patients  for  whom  use  of  the  intensive  care  unit  and  telemetry  makes  a  differ- 
ence (Griner,  1973). 

It  is  notable  that  we  found  a  significant  process-outcome  relationship  for  patients  with 
all  four  of  the  medical  diseases  studied,  but  not  for  hip  fracture.  This  may  be  because  short- 
term  mortality  occurs  less  often  for  hip  fracture  patients  than  for  the  other  study  diseases. 
Alternatively,  it  may  be  that  mortality  is  not  the  best  outcome  to  study  for  hip  fracture 
patients.  Another  possibility  is  that  the  medical  record  does  not  provide  an  adequate  data 
source  for  evaluating  surgical,  particularly  intra-operative,  processes.  Methods  for  evaluat- 
ing surgical  processes  of  care  need  further  development.  Since  the  improvements  in  process 
noted  after  the  introduction  of  prospective  payment  for  hip  fracture  patients  are  consistent 
with  improvements  in  care  seen  for  the  other  four  study  diseases,  we  believe  that  care  for  hip 
fracture  patients  post-PPS  has  in  fact  improved. 

In  summary,  our  consistent  findings  across  process  scales  and  diseases  suggest  both 
that  processes  of  care  have  improved  from  1981  to  1986  and  that  there  is  still  room  for 
further  improvement.  This  provides  evidence  that  the  implementation  of  PPS  was  not  asso- 
ciated with  a  deterioration  in  care,  even  in  areas  that  were  most  sensitive  to  the  financial 
incentives  provided  by  prospective  payment  to  decrease  the  level  of  services,  such  as  nursing 
activities  and  the  use  of  intensive  care  units.  This  is  an  encouraging  finding  and  indicates 
that  the  24  percent  decrease  in  length  of  stay  post-PPS  was  not  associated  with  deterioration 
in  the  process  of  inhospital  care.  It  is  our  hope  that  our  process  criteria  can  be  expanded  and 
updated,  both  for  our  five  diseases  and  for  other  diseases  as  well,  to  provide  an  ongoing  sys- 
tem for  monitoring  how  patients  are  treated  as  medical  care  evolves. 


V.  CHANGES  IN  QUALITY  OF  CARE  BETWEEN  1981  AND  1986 
FOR  FIVE  DISEASES  AS  MEASURED  BY  IMPLICIT  REVIEW 


Lisa  V.  Rubenstein,  M.D.,  MSPH,  Katherine  L.  Kahn,  M.D., 
Ellen  J.  Reinisch,  M.S.,  Marjorie  J.  Sherwood,  M.D., 
William  H.  Rogers,  Ph.D.,  Caren  Kamberg,  MSPH, 
David  Draper,  Ph.D.,  Robert  H.  Brook,  M.D.,  Sc.D. 


The  changes  in  the  reimbursement  system  for  Medicare  patients,  described  in  previous 
sections,  have  given  rise  to  national  concern  that  the  quality  of  medical  care  may  have 
deteriorated.  Physicians  and  patients  feared  that  the  process  of  medical  care  might  be 
altered  in  favor  of  inappropriately  low-cost  treatment  strategies.  The  state  PROs  were 
intended  to  provide  a  safeguard  against  these  trends,  but  neither  the  public  nor  the  medical 
profession  was  certain  that  PROs  could  accurately  assess  and  influence  the  quality  of  medical 
care. 

Quality  of  care  can  be  judged  either  by  implicit  or  explicit  review  of  data  from  medical 
records  (Brook,  1973;  Altman  et  al.,  1969;  and  Donabedian,  1982).  Explicit  review  relies  on 
preset  criteria,  whereas  implicit  review  depends  upon  the  practitioner's  opinion  to  generate  a 
quality  rating.  In  Sec.  IV  we  documented  that  the  quality  of  medical  care  in  the  United 
States,  when  measured  by  explicit  review  of  the  medical  record  for  five  major  diseases,  has 
generally  improved  over  the  years  between  1981  and  1986.  Implicit  review  of  the  medical 
record,  however,  is  the  current  community  gold  standard  for  making  final  judgments  about 
the  quality  of  medical  care  (Dans  et  al.,  1985).  It  is  the  standard  method  used  by  hospitals, 
professional  review  organizations,  and  insurance  companies  to  judge  quality. 

The  purpose  of  this  study  was  to  improve  implicit  review,  to  determine  the  reliability 
and  validity  of  the  improved  method,  and  then  to  use  it  to  evaluate  changes  in  the  quality  of 
medical  care  for  Medicare  patients  in  the  United  States  between  1981  and  1986.  We  report 
as  well  upon  the  relationship  between  judgments  of  quality  based  on  implicit  review  and 
those  based  on  explicit  review. 

METHODS 
Patient  Sample 

The  patient  sample  for  the  explicit  review  portion  of  this  study  was  described  above. 

All  included  medical  records  underwent  explicit  review.  Ten  percent  of  included  charts 
(a  total  of  1,366  records)  were  selected  to  undergo  implicit  review.  Stratified  random  sam- 
pling was  used,  with  oversampling  of  deaths  so  that  approximately  50  percent  of  patients 
whose  records  underwent  implicit  review  were  patients  who  had  died  in  the  hospital.  This 
sampling  plan  maximized  our  chance  of  identifying  patients  for  whom  death  occurred  as  a 
result  of  poor  quality.  In  the  analyses  reported  here,  data  have  been  reweighted  to  achieve 
representativeness  of  our  findings  for  all  the  original  14,012  patients  sampled. 
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Performing  Implicit  Review 

Before  undergoing  implicit  review,  the  inpatient  records  of  all  study  patients  were 
duplicated  and  deidentified  by  obliterating  all  names  of  patients,  hospitals,  physicians, 
nurses,  cities,  and  states.  Dates  were  retained. 

To  perform  implicit  reviews,  reviewers  were  instructed  to  examine  the  entire  medical 
record  with  the  exception  of  nursing  notes  (Rubenstein  et  al.,  1991).  Nursing  notes  were 
available  to  the  reviewer  for  use  as  needed,  but  because  of  time  constraints  were  not 
reviewed  in  their  entirety. 

Twenty-five  physician  reviewers  participated  in  the  study.  Each  of  the  five  state  PROs 
participating  in  the  study  selected  one  reviewer  per  disease,  but  each  reviewer  reviewed 
records  from  all  states.  We  randomly  assigned  records  to  reviewers.  No  reviewer  reviewed 
patients  from  more  than  one  of  the  five  study  diseases.  All  reviewers  were  board  certified  in 
appropriate  specialties.  Internists  reviewed  records  of  patients  hospitalized  with  congestive 
heart  failure,  cardiologists  reviewed  acute  myocardial  infarction  records,  pulmonologists 
reviewed  pneumonia  records,  neurologists  reviewed  cerebrovascular  accident  records,  and 
orthopedists  reviewed  hip  fracture  records. 

Implicit  review,  as  performed  in  this  study,  required  that  reviewers  be  trained  in  the 
use  of  a  structured  review  form  with  accompanying  written  guidelines.  Reviews  were  bud- 
geted at  30  minutes  each;  reviewers  were  instructed  to  keep  to  this  time  limit,  although  some 
reviews  might  require  less  time  and  some  more.  We  budgeted  training  at  12  hours  per  physi- 
cian. In  training,  we  specifically  avoided  trying  to  change  reviewers'  opinions  about  what 
ideal  or  acceptable  quality  of  care  would  look  like;  rather,  our  training  addressed  how  dif- 
ferent data  sources  within  the  medical  record  can  be  used  to  judge  quality,  how  elements  of 
the  record  that  might  be  used  to  judge  quality  could  be  weighted,  and  what  is  meant  by  the 
terms  excellent,  medium,  and  poor  care  (Rubenstein  et  al.,  1991;  and  Kahn  et  al.,  1989). 

Initial  training  of  physician  reviewers  was  carried  out  during  one  three-hour  small- 
group  session.  Following  the  session,  each  physician  reviewed  two  training  records  and  par- 
ticipated in  one  one-half  hour  telephone  call  with  study  physicians.  Physicians  then 
reviewed  three  more  records  on  their  own  at  home;  these  records  were  discussed  during  a 
three-hour  conference  call  involving  all  five  physicians  and  two  study  investigators. 

Reviewers  were  taught  during  training  to  anchor  their  ratings.  For  most  items, 
reviewers  were  asked  to  use  the  lowest  response  category  when  the  care  given  would  have 
been  highly  likely  to  contribute  to  a  bad  outcome,  whether  or  not  a  bad  outcome  actually 
occurred  in  that  case.  For  rating  the  quality  of  a  physician's  history  or  clinical  assessment, 
raters  were  instructed  to  use  the  lowest  response  category  ("very  poor")  if  the  rater,  when 
asked  to  see  the  patient  at  midnight,  would  have  to  start  from  scratch  in  his  evaluation.  A 
judgment  of  "adequate"  meant  that  most  essential  historical  and  assessment  observations 
were  included,  but  additional  data  might  be  required  for  optimal  diagnosis  and  treatment. 
"Excellent"  meant  that  all  necessary  data  were  present.  "Good"  was  somewhat  better  than 
adequate  and  "poor"  somewhat  worse. 

Reviewers  were  asked  to  judge  urban,  rural,  teaching,  and  nonteaching  hospitals 
according  to  the  same  standard.  They  were  asked  not  to  adjust  their  ratings  according  to 
their  guess  about  the  type  of  hospital  from  which  a  record  came.  Reviewers  were  asked  to 
rate  care  as  inadequate  (i.e.,  as  "poor"  or  "very  poor")  when  that  care  did  not  meet  a  level 
achievable  by  most  hospitals,  and  not  to  judge  care  as  inadequate  solely  for  failure  to  perform 
extraordinary  or  controversial  procedures. 
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Reviewers  were  asked  to  take  account  of  DNR  orders.  They  were  asked  to  judge  care  for 
these  patients  according  to  their  own  implicit  standards  for  the  care  of  DNR  patients. 
Reviewers  were  asked  not  to  second-guess  the  medical  records,  in  terms  of  the  level  of 
aggressiveness  of  treatment  aspired  to  by  the  physicians  caring  for  the  patient,  when  no 
explicit  statement  about  low  aggressiveness  of  care  was  present  in  the  record. 

The  Structured  Implicit  Review  form,  shown  at  the  end  of  this  section,  included  27 
items  and  asked  the  physician  reviewers  to  rate  specific  aspects  of  the  following:  the  process 
of  physician  and  nursing  care;  the  appropriateness  of  use  of  hospital  services;  patient  prog- 
nosis; treatability  of  the  patient's  condition;  preventability  of  death  when  it  occurred;  the 
quality  of  the  outcome;  and  an  overall  assessment  of  the  quality  of  care  delivered  during  the 
hospitalization.  Ratings  were  based  on  Likert  scales;  a  five-point  scale  from  very  poor  to 
excellent  was  used  for  most  of  the  items.  We  used  the  same  review  form  for  patients  with 
congestive  heart  failure,  myocardial  infarction,  and  pneumonia.  We  modified  the  form 
slightly  for  hip  fracture  and  cerebrovascular  accident. 

To  determine  interrater  reliability,  a  randomly  selected  sample  of  about  25  percent  of 
all  records  were  reviewed  by  two  reviewers,  and  3  percent  were  reviewed  by  all  five 
reviewers. 

Additional  Data  Sources 

In  addition  to  performing  implicit  reviews  on  the  medical  records,  we  used  data  from 
the  previously  described  explicit  reviews  of  these  records  (Sec.  IV).  To  evaluate  the  actual 
outcomes  of  care  of  the  patients  whose  care  was  assessed  implicitly,  we  examined  mortality 
rates  30  days  after  admission.  We  were  able  to  find  accurate  postadmission  mortality  data 
from  HCFA's  Health  Insurance  Master  file  for  92  percent  of  our  study  sample  (Sec.  VII). 

Statistical  Methods 

To  facilitate  analysis,  we  developed  process  scales  and  an  overall  quality  of  care  scale. 
We  used  clinical  judgment  to  group  process  items  into  scales.  Process  scales  were  created 
from  the  19  items  listed  in  questions  1-5  on  the  implicit  review  form.  The  overall  quality  of 
care  scale  was  based  on  answers  to  questions  9  and  10.  We  used  factor  analysis  to  determine 
whether  our  process  groupings  made  sense  psychometrically;  final  scales  were  quite  similar 
to  our  initial  clinically  developed  scale  groupings.  We  moved  an  item  from  one  scale  to 
another  when  a  high  correlation  with  the  other  factor  indicated  strongly  that  the  item 
belonged  in  that  grouping. 

Overall  Quality  Scale.  The  overall  quality  of  care  scale  consisted  of  two  items:  "Con- 
sidering everything  you  know  about  the  patient,  please  rate  overall  quality  of  care"  with  five 
response  categories  ranging  from  extreme,  above  standard,  to  extreme,  below  standard;  and 
"Would  you  send  your  mother  to  these  physicians  in  this  hospital?"  with  four  response 
categories  ranging  from  definitely  yes  to  definitely  no.  Responses  to  these  questions  were 
added  together  to  form  an  eight-point  overall  quality  of  care  scale  from  2  (both  questions 
answered  with  the  highest  possible  rating  of  1)  to  9  (worst  care).  Cases  reviewed  by  multiple 
reviewers  were  assigned  the  mean  adjusted  score  across  reviewers.  We  then  divided  the 
overall  quality  of  care  scale  into  four  levels  of  care:  (1)  "very  poor  care,"  a  score  of  7.5  to  9;  (2) 
"poor  care,"  a  score  of  6.5-7.4;  (3)  "good  care,"  a  score  of  3.5-6.4;  and  (4)  "very  good  care,"  a 
score  of  2-3.4.  Poor  care  or  very  poor  care  means  that  responses  to  both  of  the  overall  quality 
questions  were  rated  in  the  "below  standard"  or  "probably  would  not  send  my  mother"  range. 
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Reliability.  We  hypothesized  that  some  reviewers  might  judge  care  more  harshly,  and 
some  more  leniently,  than  others.  We  tested  for  the  significance  of  this  "reviewer  effect" 
using  a  one-way  analysis  of  variance.  To  perform  this  adjustment,  we  calculated  the  group  of 
reviewers'  mean  score  for  a  question  or  scale  across  patients,  compared  it  to  the  individual 
reviewer's  mean  score  for  that  question,  and  adjusted  the  individual  reviewer's  scores  for  all 
patients  up  or  down  proportionally  to  match  the  group  mean.  We  then  used  the  adjusted 
scores  in  our  analyses. 

We  also  used  analysis  of  variance  to  assess  interrater  reliability  because  we  use  ordered 
scales.  For  example,  a  5  on  one  of  our  scales  is  not  very  different  from  a  6,  but  very  different 
from  a  9;  in  assessing  interrater  reliability,  analysis  of  variance  takes  these  relative  disagree- 
ments into  account  appropriately.  Before  calculating  interrater  reliability,  we  adjusted 
scores  for  reviewer  effects  as  described  above. 

Components  of  Variance.  We  used  components  of  variance  analysis  to  understand 
reasons  for  disagreement  among  multiple  reviewers  of  the  same  case,  and  to  examine  the 
relationship  between  this  disagreement  and  the  conclusions  that  could  be  drawn  about  the 
true  quality  differences  between  patients  assigned  particular  quality  ratings.  The  variance 
analysis  of  the  distribution  of  cases  between  very  good,  good,  poor,  and  very  poor  care 
included  (1)  the  "reviewer  effect"  discussed  above,  (2)  differences  of  opinion  among  reviewers, 
and  (3)  actual  differences  in  the  medical  care  delivered  to  the  patient.  We  are  clearly  most 
interested  in  the  last  component  of  the  variance.  We  used  the  variance-components  model  to 
adjust  results  of  the  overall  quality  scale  for  the  reviewer  effects  and  for  differences  of  opin- 
ion among  reviewers.  These  components  are  mathematically  related  to  imperfect  reliability. 
The  figures  generated  by  this  method  provide  a  conservative  estimate  of  the  amount  of  care 
judged  to  be  inadequate. 

Evaluating  the  Relationship  Between  Quality  of  Care  and  30-Day  Death.  We 
used  linear  regression  to  evaluate  whether  poor  quality  of  care  measured  implicitly  by  the 
overall  quality  scale  predicted  death  within  30  days,  after  adjusting  for  patient  sickness  at 
admission  (Sec.  III).  (Logistic  regression  produced  similar  results.) 

We  also  used  linear  regression  to  determine  whether  reviewers  assigned  quality  scores 
differently  when  patients  died  in,  as  compared  to  outside,  the  hospital.  To  account  for  the 
difference  between  earlier  and  later  deaths,  we  controlled  for  the  number  of  days  between 
hospital  admission  and  day  of  death  in  our  analyses. 

RESULTS 
Patient  Sample 

A  total  of  278  records  for  congestive  heart  failure,  275  records  for  acute  myocardial 
infarction,  273  records  for  pneumonia,  270  records  for  cerebrovascular  accident,  and  270 
records  for  hip  fracture  were  reviewed.  This  represented  93  percent  of  records  randomly 
selected  for  implicit  review.  Of  the  1,366  records  reviewed,  993  were  reviewed  once,  333  were 
reviewed  twice,  33  were  reviewed  five  times,  and  seven  (hip  fracture)  records  were  reviewed 
four  times.  To  evaluate  our  sampling,  we  compared  the  implicit  review  sample  to  the  explicit 
sample,  which  is  known  to  be  nationally  representative  (Sec.  II).  The  implicit  and  explicit 
samples  did  not  differ  significantly  in  terms  of  age,  sex,  race,  or  type  of  hospital  from  which 
they  came  (Table  5.1). 
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Table  5.1 

DEMOGRAPHIC  CHARACTERISTICS  OF  PATIENTS  IN 
IMPLICIT  AND  EXPLICIT  REVIEW  SAMPLES 
(In  percent) 


Review  Sample 


Implicit 

Explicit 

Characteristics 

(n=l,366) 

(n=14,012) 

Patient 

Age  *  80 

43 

41 

Female 

54 

57 

Non-white 

16 

19 

Hospitals 

Rural 

20 

21 

Any  teaching 

36 

35 

County 

13 

14 

Serves  high  percentage 

Medicaid  patients 

16 

19 

Reliability 

Process  scales  and  the  overall  quality  scale,  with  sample  items  and  reliabilities,  are 
listed  in  Table  5.2.  Cronbach's  alpha  or  interitem  reliabilities  were  between  0.8  and  0.9. 
Interrater  reliabilities  for  scales  were  mostly  between  0.4  and  0.7,  a  range  considered  ade- 
quate for  comparing  groups  of  people  (Koran,  1975).  These  reliabilities  are  in  general  better 
than  those  previously  obtained  by  unstructured  implicit  review  (Brook,  1973;  Dubois  et  al., 
1987;  and  Richardson,  1972).  All  scores  are  adjusted  for  the  reviewer  effect  (discussed  in 
Methods,  above),  which  turned  out  to  be  small  but  significant,  accounting  for  about  4  percent 
of  the  variance  between  cases. 

Overall  Quality  Scale 

The  Pearson  correlation  coefficient  between  the  two  questions  on  the  overall  quality  of 
care  scale  (see  Table  5.2)  was  0.86.  The  overall  quality  scale  also  closely  correlated  with  the 
four  process  scales,  and  the  preventable  death  item  (Table  5.3).  For  example,  the  high  corre- 
lation between  the  "physician  assessment  and  plan"  and  the  overall  quality  scale  (r  =  0.80) 
indicated  that  reviewers  were  much  more  positive  in  their  assessments  of  the  overall  quality 
of  care  when  the  reviewers  perceived  that  the  patient  was  properly  assessed  by  the  physician 
early  in  the  hospitalization.  The  correlation  of  the  overall  quality  of  care  scale  with  nurse 
assessment,  however,  was  low  (r  =  0.23),  indicating  that  our  overall  quality  score  did  not 
strongly  reflect  the  reviewer's  judgment  of  the  initial  nursing  assessment.  This  was  as 
expected,  because  we  did  not  require  review  of  all  nursing  notes.  The  correlation  with 
patient  prognosis  was  also  low  (r  =  0.01),  indicating  that  physicians  were  not  confusing  qual- 
ity of  care  ratings  with  patient  sickness  at  admission. 

Overall,  responses  to  the  more  detailed  process  items  and  scales  accounted  for  about  72 
percent  of  the  variance  in  the  overall  quality  scale.  This  finding  indicated  that  the  overall 
quality  scale  reflected  reviewer's  judgments  about  the  multiple  components  of  hospital  care 
listed  elsewhere  on  the  review  form  and  could  appropriately  be  used  as  a  summary  measure. 
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Table  5.2 

RELIABILITY  OF  IMPLICIT  REVIEW  SCALES  AND  ITEMS  ACROSS  FIVE  DISEASES 


Implicit  Review 
Scales 


Range  of 
Number 

of  Items  in   

the  Scale  Mean 


Cronbach's 
Interitem 
Reliability 


Interrater 
Reliability 


Range      Mean  Range 


Example  Questions 
from  Structured 
Implicit  Review  Form 


Physician  function  and 
chronic  disease: 

Physician's  initial 
assessment  of  function, 
habit,  and  chronic  disease 

Physician  assessment  and 
plan: 

Physician's  initial 
assessment  of  the 
acute  disease 

Laboratory  evaluation: 

Use  of  laboratory  tests 
during  the  hospitalization 


Treatments: 

Initial  treatment  plan, 
orders,  and  use  of 
medical  therapies 

Overall  quality: 

Overall  quality  of 
care  during  the 
hospitalization 


Length  of  hospital 
stay  (for  patients  who 
survived  hospitalization) 

Preventability  of  death 
(for  patients  who  died) 

Quality  of  the  outcome 
compared  with  that  expected 
based  on  the  patient's 
health  status  on  day  1 
of  hospitalization 

Life  expectancy  based  on 
the  patient's  presentation 
on  day  1  of  hospitalization 

Treatability  of  the  disease, 
assuming  adequate  to 
optimal  care  by  |he  physician 
and  the  hospital 

Nurse  assessment  of  prior 
and  chronic  disease,  function, 
habits,  psycho-social  status 


2-4         0.88     (0.82-0.92)     0.69  (0.65-0.74) 


2-4        0.88     (0.67-0.94)    0.59  (0.25-0.69) 


3-4         0.80     (0.74-0.88)     0.53  (0.22-0.74) 


3-6         0.82      (0.69-0.88)     0.49  (0.36-0.65) 


2         0.92      (0.87-0.94)     0.54  (0.42-0.66) 


1  NA 

1  NA 

1  NA 

1  NA 

1  NA 

1  NA 


0.30  (0.01-0.42) 

0.28  (0.04-0.50) 
0.60  (0.58-0.73) 


Rate  physician  documentation 
of  prior  and  chronic  disease 
from  "excellent"  to  "very  poor" 


Rate  completeness  of  initial 
physician  data-gathering  from 
"excellent"  to  "very  poor" 

Rate  use  of  venous  blood  tests, 
urinalyses,  and  sputum  analyses 
from  "definitely  appropriate"  to 
"definitely  inappropriate" 


Rate  initial  physician's 
treatment  plan  and  orders  from 
"excellent"  to  "very  poor" 


Considering  everything  you  know 
about  this  patient,  please  rate 
overall  quality  of  care,  from 
"extreme,  above  standard"  to 
"extreme,  below  standard"  and 
"definitely  would  send  my  mother 
to  these  physicians  in  this  hospital" 
to  "definitely  would  not" 

"Definitely  appropriate"  to 
"definitely  inappropriate" 

"Definitely  preventable"  to 
"definitely  not  preventable" 

"Much  better  than  expected"  to 
"much  worse  than  expected" 


0.66    (0.58-0.73)   "<  1  month"  to  ">  1  year" 


0.57  (0.34-0.69) 


"Very  effective"  to  "very 
ineffective" 


0.37    (0.20-0.50)   "Excellent"  to  'Very  poor" 


^Interitem  reliability  is  not  applicable  for  single  items. 
Item  not  present  on  review  form  for  hip  fracture. 
Item  not  present  on  review  form  for  cerebrovascular  accident. 
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Table  5.3 

PEARSON  CORRELATION  COEFFICIENTS  OF  INDIVIDUAL  IMPLICIT 
REVIEW  SCALES  AND  ITEMS  WITH  THE  OVERALL 
QUALITY  OF  CARE  SCALE8 


Implicit  Review  Scales 

Mean  for 

Range  of  Values 

and  Items'3 

5  Diseases 

Across  5  Diseases 

Physician  functional  and  chronic 

0.60 

(0.43,  0.68) 

disease  assessment 

Physician  assessment  and  plan 

0.80 

(0.59,  0.85) 

Laboratory  evaluation 

0.59 

(0.40,  0.74) 

Treatment 

0.69 

(0.49,  0.77) 

Nurse  assessment 

0.23 

(0.10,  0.36) 

Quality  of  the  outcome 

0.36 

(0.30,  0.41) 

Length  of  stay 

0.51 

(0.39,  0.63) 

Preventable  death 

-0.65 

(-0.49,  -0.66) 

Patient  prognosis  at  admission 

0.01 

(-0.01,  0.12) 

All  scales  and  items  except  preventable  death  were  scored  so  that  a 
positive  correlation  indicates  that  better  overall  quality  scores  are  associ- 
ated with  better  performance  on  the  scales  in  question;  e.g.,  better  assess- 
ments, better  outcomes,  or  more  appropriate  length  of  stay. 

See  Table  5.2  for  definitions. 


To  further  validate  our  structured  implicit  review,  we  examined  the  relationship 
between  the  process  of  care  and  mortality,  after  controlling  for  admission  sickness.  We  found 
(Table  5.4)  that,  at  each  level  (quartile)  of  sickness  at  admission,  death  within  30  days  of  hos- 
pital admission  increased  as  quality  of  care  measured  by  implicit  review  decreased.  For 
instance,  of  patients  in  the  third  quartile  of  sickness,  13  percent  who  experienced  good  or 
very  good  process  died,  whereas  34  percent  who  experienced  very  poor  process  died.  Aggre- 
gating across  diseases  and  sickness  quartiles,  the  relative  risk  of  30-day  death  for  patients 
with  very  poor  care  as  compared  with  all  other  patients  is  2.08  (p  <  0.01). 

These  findings  were  confirmed  in  multiple  regressions  in  which  our  implicit  quality  of 
care  measure  significantly  predicted  outcome,  even  after  adjusting  for  both  implicit  and 
explicit  measures  of  sickness  (p  <  0.01). 

These  findings  also  held  for  cerebrovascular  accident,  but  not  for  other  diseases,  when 
we  examined  by  regression  the  relationship  between  the  implicit  quality  of  care  measure  and 
out-of-hospital  death  within  30  days,  although  the  trends  for  all  diseases  for  in-  and  out-of- 
hospital  deaths  were  similar  (i.e.,  patients  with  poor  quality  scores  had  an  increased  risk  of 
death).  The  analysis  of  only  out-of-hospital  deaths  eliminated  the  potential  bias  of  reviewers 
rating  the  quality  of  care  lower  for  patients  who  died  inhospital.  However,  this  analysis, 
which  excluded  patients  with  inhospital  death,  excluded  about  82  percent  of  the  deaths 
occurring  within  30  days  of  admission.  This  greatly  decreased  the  strength  of  the  process- 
outcome  relationship. 

We  found  no  significant  effect  of  inhospital  compared  with  out-of-hospital  death  on  the 
overall  quality  score  using  linear  regression.  This  indicates  that  reviewers  were  not  system- 
atically downrating  care  given  to  patients  who  died  during  their  hospitalization  rather  than 
after  it. 
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Table  5.4 

PERCENTAGE  OF  PATIENTS  WHO  DIED  WITHIN  30  DAYS  OF 
HOSPITALIZATION,  BY  OVERALL  QUALITY  OF  CARE 
RECEIVED  AND  LEVEL  OF  SICKNESS  AT  ADMISSION6-1* 


Quartile 


I 

II 

III 

IV 

Quality  of  Care 

Least  Sick 

Most  Sick 

All 

by  Implicit  Review 

(n=294) 

(n=301) 

(n=300) 

(n=302) 

(n=l,197) 

Very  good  (n=189) 

1 

5 

13 

46 

17 

Good  (n=675) 

3 

6 

13 

34 

14 

Poor (n=164) 

2 

4 

8 

49 

13 

Very  poor  (n=169)  (2) 

6* 

17** 

34** 

55** 

30** 

Total  patients  (n=l,197) 

3 

7 

15 

41 

17 

Relative  risk  of  30-day 

death:  very  poor  compared 

2.43* 

3.09** 

2.79** 

1.42** 

2.08** 

with  all  others0 

aSickness  at  admission  is  measured  explicitly  in  Sec.  III.  We  constructed 
severity  quartiles  for  all  five  diseases  by  dividing  patients  into  four  equal  groups 
ranging  from  the  least  sick  25  percent  to  the  most  sick  25  percent. 

We  found  no  evidence  that  reviewers  systematically  downrated  quality  for 
patients  who  died  in  the  hospital.  Stratifying  by  day  of  death  inhospital  deaths, 
and  out-of-hospital  deaths  had  equivalent  quality  scores.  The  sample  size  for  out- 
of-hospital  deaths  was  small,  but  the  trend  was  the  same  as  for  inhospital  deaths; 
i.e.,  patients  with  poor  quality  scores  had  an  increased  risk  of  death  (p  =  0.07). 

c30-day  death  rate  for  patients  with  very  poor  care  differs  from  the  correspond- 
ing rate  for  all  other  patients  by  an  amount  whose  significance  is  indicated  by 
asterisks:  *  =  p  <  0.05,  **  =  p  <  0.01  (i.e.,  relative  risks  are  significantly  different 
from  1). 


Comparison  Between  Implicit  and  Explicit  Reviews 

To  compare  implicit  review  and  explicit  review  evaluations  of  quality  of  care,  we  used 
the  four  levels  of  care  measured  by  our  overall  quality  of  care  scale  (see  Methods,  above). 
Explicit  scales  are  scored  from  0  to  100  based  on  the  percentage  of  applicable  explicit  criteria 
adhered  to  in  the  patient's  medical  record.  A  higher  score  indicates  better  process  by  explicit 
criteria  (Sec.  IV). 

Explicit  process-scale  scores  were  consistently  lower  (worse)  for  the  group  of  patients 
who  had  received  poor  or  very  poor  care  as  judged  by  implicit  review.  Results  of  comparisons 
with  three  of  the  explicit  scales  are  presented  in  Table  5.5.  "Physician  Process  Day  1  and 
Day  2"  measures  whether  any  physician  recorded  specified  information  about  the  history  and 
physical  examination  on  days  1  or  2  of  the  hospitalization.  "Physician  Process  Day  3"  asks 
whether  any  physician  recorded  such  information  on  day  3  of  the  hospitalization.  "Abnormal 
Lab  Rechecked"  asks  whether  highly  abnormal  laboratory  results,  such  as  a  potassium  <  3.0 
mmol/L,  were  ever  rechecked  during  the  hospitalization. 

For  "Physician  Process  Day  1  and  Day  2,"  and  "Day  3,"  the  very  good  and  good  care 
groups  based  on  implicit  review  had  significantly  different  explicit  scores  from  each  other 
(p  <  0.001)  and  from  the  poor  and  very  poor  groups.  For  the  "Abnormal  Lab  Rechecked" 
scale,  good  and  very  good  could  not  be  distinguished  on  the  explicit  scale  from  each  other  but 
scored  significantly  higher  than  did  poor  or  very  poor  cases  (p  <  0.01).  The  poor  and  very 
poor  groups,  on  the  other  hand,  could  not  be  distinguished  from  each  other  based  on  explicit 


81 


Table  5.5 

COMPARISON  BETWEEN  IMPLICIT  AND  EXPLICIT  PROCESS-SCALE 
SCORES  FOR  FIVE  DISEASES 
(Average  process  on  explicit  scales,  0-100)8 


Overall  Implicit 
Quality  Scale 

Physician  Process 
Day  1,  2b 
(n=  1,849) 

Physician  Process 
Day  3c'd 
(n=l,316) 

Abnormal 
Lab  Rechecked6 
(n=909) 

Very  good  care 

88 

76 

73 

Good  care 

75 

59 

68 

Poor  care 

69 

48 

57 

Very  poor  care 

63 

47 

55 

8A  higher  score  indicates  better  performance  for  explicit  review  scales. 
Significance  of  differences  for  all  pairwise  comparisons  is  p  <  0.001. 
jThis  scale  was  not  measured  for  hip  fracture. 

Significance  of  differences  for  all  pairwise  comparisons  is  p  <  0.001 
except  for  poor  compared  with  very  poor,  which  is  not  significant. 

Significant  differences  for  very  good  compared  with  poor  or  very  poor, 
and  for  good  compared  with  very  poor  (p  <  0.01). 

process  scores.  These  results  held  when  patients  discharged  alive  and  those  discharged  dead 
were  analyzed  separately. 

Overall,  out  of  the  67  possible  comparisons  between  explicit  process  scores  and  implicit 
review  scores,  54  (81  percent)  had  higher  scores  for  cases  that  were  rated  good  or  very  good 
than  poor  or  very  poor,  nine  (13  percent)  of  the  scales  had  equivalent  scores  for  the  two 
groups,  and  the  remaining  four  (6  percent)  had  lower  (worse)  explicit  process  scores  among 
the  cases  rated  better  on  implicit  review.  This  pattern  of  results  is  highly  significant  (p  < 
0.001). 

Overall  Level  of  Quality  of  Care  and  Pre-Post  PPS  Differences 

Reviewers  judged  the  quality  of  care  for  the  majority  of  patients  in  our  sample  to  be 
good  (Table  5.6),  combining  all  study  years.  Many  more  cases  (82  percent,  for  all  five 
diseases  combined)  were  considered  to  be  good  or  very  good  than  were  judged  to  be  poor  or 
very  poor  (18  percent).  For  congestive  heart  failure,  however,  27  percent  of  the  patients  were 

Table  5.6 

QUALITY  OF  MEDICAL  CARE  FOR  FIVE  DISEASES  AS  MEASURED  BY  IMPLICIT  REVIEW8 

Distribution  of  Patients  on  the  Overall 
Quality  of  Care  Scale,  % 


Disease 

No.  of 
Patients 

Very  Good 

Good 

Poor 

Very  Poor 

%  Poor  or 
Very  Poor 

Congestive  heart  failure 

278 

4 

68 

19 

8 

27 

Acute  myocardial  infarction 

275 

10 

69 

15 

7:V 

23 

Pneumonia 

273 

8 

79 

11 

2 

13 

Cerebrovascular  accident 

270 

12 

64 

15 

9 

24 

Hip  fracture 

270 

3 

93 

4 

0 

4 

All  diseases  combined 

1,366 

7 

75 

13 

5 

18 

Reviewer  effects  removed  (see  the  text). 
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judged  to  have  received  poor  or  very  poor  care;  for  cerebrovascular  accident,  24  percent 
received  poor  or  very  poor  care;  for  acute  myocardial  infarction,  23  percent  received  poor  or 
very  poor  care;  and  for  pneumonia,  13  percent  received  poor  or  very  poor  care.  Patients  with 
hip  fracture  were  least  often  judged  to  receive  poor  or  very  poor  care  (4  percent). 

For  all  diseases,  reviewers  judged  the  quality  of  care  from  the  1985-1986  time  period 
more  favorably  than  that  in  the  1981-1982  time  period  (Table  5.7).  This  difference  was  sta- 
tistically significant  for  acute  myocardial  infarction  and  cerebrovascular  accident  (p  <  0.001), 
and  for  congestive  heart  failure  (p  <  0.01).  The  trend  was  not  significant  for  pneumonia  (p  < 
0.10)  or  for  hip  fracture  (p  <  0.20).  Virtually  all  scales  and  items  on  the  form  were  rated 
higher  in  the  post-PPS  time  period,  1985-1986,  and  most  pre-  post-PPS  differences  were 
significant  at  the  p  <  0.001  level  when  we  aggregated  our  findings  across  the  five  diseases. 
Instability  at  discharge,  however,  was  assessed  by  reviewers  as  significantly  worse  during 
1985-1986  than  during  1981-1982  (Table  5.8). 

Physicians  judged  length  of  stay  to  be  more  appropriate  overall  in  1985-1986  than  in 
1981-1982  for  three  diseases  (acute  myocardial  infarction,  pneumonia,  and  cerebrovascular 
accident),  and  about  equally  appropriate  for  two  diseases  (congestive  heart  failure  and  hip 
fracture).  The  distribution  of  inappropriateness  between  stays  that  were  inappropriately  too 
long  and  inappropriately  too  short  also  changed  from  pre-  to  post-PPS,  however,  with  fewer 
of  the  inappropriate  stays  during  the  1985-1986  period  being  due  to  inappropriately  long 
stays,  and  more  being  due  to  inappropriately  short  stays.  This  was  particularly  true  of 
congestive  heart  failure,  where  32  percent  of  patients  were  judged  to  have  been  discharged 
too  soon  during  the  later  period  compared  with  24  percent  in  the  earlier  period,  and  for  hip 
fracture,  where  23  percent  of  patients  were  so  judged  in  the  later  period,  compared  with  12 
percent  in  the  earlier  period.  For  these  two  diseases,  instability  at  discharge  post-PPS  was 
judged  to  be  present  in  nearly  half  of  the  patients  who  were  discharged  too  soon.  Overall,  the 
fraction  of  hospitalizations  rated  as  both  too  short  and  as  resulting  in  patients  discharged 
unstable  almost  doubled  from  pre-  to  post-PPS  (4.0  percent  to  7.1  percent,  p  <  0.05). 

Table  5.7 

PERCENTAGE  OF  PATIENTS  WITH  POOR  OR  VERY  POOR  CARE  PRE-  AND  POST-PPSa 

No.  of  patients 
(Pre-  and 


Post-PPS 

Pre-PPS 

Post-PPS 

Difference 

Disease 

Combined) 

(n=669) 

(n=697) 

(Post-Pre)b 

Congestive  heart  failure 

278 

35 

22 

-13* 

Acute  myocardial  infarction 

275 

33 

13 

-20** 

Pneumonia 

273 

17 

10 

-7+ 

Cerebrovascular  accident 

270 

36 

16 

-20** 

Hip  fracture 

270 

5 

2 

-3 

Five  diseases  combined 

1,366 

25 

12 

-13** 

Reviewer  effects  removed. 

Significance  of  pre-  post-PPS  difference  is  indicated  by  the  following  symbols:  +  =  p  < 
0.10,  *  =  p  <  0.01,  **  =  p  <  0.001.  The  pattern  of  significance  takes  into  account  not  only  the 
size  of  the  pre-  post-PPS  difference  but  also  reliability. 
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Table  5.8 

CHANGES  IN  THE  PERCENTAGE  OF  PATIENTS  JUDGED  TO  HAVE  INAPPROPRIATE 
LENGTH  OF  STAY  FOR  FIVE  DISEASES  PRE-  AND  POST-PPSa 


Probably  or 

Definitely  Too  Short  and 

Inappropriate  Too  Too         Unstable  at 

Length  of  Stay  Long  Short  Discharge15 


Pre 

Post 

Pre 

Post 

Pre 

Post 

Pre 

Post 

Congestive  heart  failure  (n=247) 

34 

35 

10 

3* 

24 

32 

12.5 

17.3 

Acute  myocardial  infarction  (n=212) 

30 

11* 

16 

3* 

14 

8 

2.1 

0.0 

Pneumonia  (n=235) 

16 

12 

10 

5* 

6 

7 

0.6 

5.1* 

Cerebrovascular  accident  (n=216) 

42 

25* 

32 

17* 

10 

8 

0.6 

1.6 

Hip  fracture  (n=258) 

27 

28 

15 

5* 

12 

23* 

3.6 

9.4 

Five  diseases  combined  (n=l,168) 

30 

23* 

17 

7* 

13 

17 

4.0 

7.1* 

NOTE:  *  p  <  0.05  for  pre-  post-PPS  comparison. 
^Includes  only  patients  discharged  alive. 

The  percentages  represent  the  proportion  of  all  patients  with  the  disease  (e.g.,  n=247  for 
heart  failure)  who  were  judged  both  to  be  unstable  and  to  have  had  a  too-short  stay. 


DISCUSSION 

Over  the  past  20  years,  implicit  review  of  the  medical  record  by  physicians  has  been  the 
community  gold  standard  forjudging  medical  care  (Dans  et  al.,  1985).  Research  has  demon- 
strated that  this  approach  to  quality  of  care  assessment  has  value  (Dubois  et  al.,  1987; 
Richardson,  1972;  Morehead  et  al.,  1971;  Payne,  1967;  Peters,  1972;  Sheps,  1955;  and  Fine  et 
al.,  1971),  but  few  studies  have  rigorously  measured  the  performance  of  the  implicit  review 
technique  (Brook,  1973;  Dubois  et  al.,  1987;  and  Richardson,  1972).  We  have  retooled  this 
classical  method  for  performing  quality  review  and  have  found  that,  in  terms  of  reliability, 
our  structured  approach  to  implicit  review  performs  as  well  as  many  other  clinical  measure- 
ments that  physicians  perform,  such  as  determining  the  presence  or  absence  of  the  dorsalis 
pedis  pulse  (39  percent  agreement)  or  determining  whether  an  electrocardiogram  is  abnor- 
mal (84  percent  agreement)  (Koran,  1975).  We  have  also  demonstrated  a  process-outcome 
link  with  implicit  review  (e.g.,  higher  scores  on  implicit  review  lead  to  better  outcomes).  We 
obtained  our  reliabilities  and  process-outcome  link  in  a  nationally  representative,  and  there- 
fore diverse,  sample  of  physicians  and  patients  (Wennberg  and  Gittelsohn,  1973;  and  Sec.  II). 
Reliabilities  were  sufficiently  high  to  allow  meaningful  comparisons  between  groups  of 
patients,  although  a  single  review  may  not  be  reliable  enough  to  accurately  judge  whether  an 
individual  patient  received  poor  care. 

We  compared  implicit  and  explicit  review  to  determine  whether  these  two  methods  mea- 
sured the  same  quality  of  care  constructs,  or  whether  they  measured  entirely  different 
aspects  of  the  quality  of  medical  care.  Happily,  the  two  methods  appear  to  measure  similar 
concepts. 

When  we  used  implicit  review  to  judge  the  quality  of  medical  care  delivered  to  a 
nationally  representative  sample  of  patients  with  the  five  diseases  we  selected  for  study,  we 
found  that  although  most  care  is  judged  to  be  good  or  very  good,  a  significant  proportion  of 
care  was  judged  to  be  poor  or  very  poor.  Associated  with  the  institution  of  prospective  pay- 
ment and  implicit  review  of  medical  records  by  PROs,  the  proportion  of  care  rated  poor  or 
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very  poor  decreased  between  1981-1982  and  1985-1986  from  about  one-quarter  of  patients  to 
about  one-eighth  of  patients.  A  similar  result  is  reported  in  Sec.  IV  when  explicit  review  is 
used. 

Many  fewer  patients  were  judged  to  have  been  kept  in  the  hospital  too  long  during  the 
later  time  period.  Of  concern,  however,  is  the  increasing  number  of  patients  judged  to  have 
been  discharged  too  soon  and,  in  particular,  with  significant  instability  at  discharge.  A  simi- 
lar increase  in  patients  discharged  unstable  is  reported  in  Sec.  VI,  using  explicit  review  cri- 
teria. 

In  summary,  structured  implicit  review  performed  well  as  a  method  for  measuring  the 
quality  of  medical  care  delivered  to  Medicare  patients.  This  is  good  news  for  PROs,  who  per- 
form the  bulk  of  quality  reviews  nationally  and  use  implicit  review  to  do  so.  The  study's 
optimistic  result  that  the  quality  of  medical  care  has  improved  post-PPS  should  serve  as 
encouraging  news  that  hospitals,  physicians,  and  other  health  professionals  continue  to  put 
patient  needs  first.  However,  substantial  improvement  is  still  possible,  and  the  increasing 
rate  of  unstable  discharges  found  by  both  implicit  and  explicit  review  is  of  concern. 
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Card  001  1-3/ 


DRG/QC  STUDY 
IMPLICIT  REVIEW  FORM 

(For  Congestive  Heart  Failure,  Acute  Myocardial  Infarction  and  Pneumonia) 


se  ID: 


Review  Date 


Month 


Day 


4-8/ 
9-14/ 


Year 


Please  rate  the  quality  of  physician  and  nurse  documentation  of  each  of  the  following: 
patient's  prior  and  chronic  disease,   functional  status,  habits,  and  psychosocial  status 
prior  to  the  current  acute  illness. 

very 

excellent    good    adequate    poor  poor 
(1)  (2)         (3)         (4)  (5) 

a)  physician  documentation 

of  prior  and  chronic  disease            15/ 


b)  physician  documentation 

of  functional  status  (e.g. , 

ambulation)            16/ 


c)  physician  documentation  of 
habits  (e.g.,  alcohol, 

smoking,  diet)            17/ 


d)  physician  documentation 
of  psychosocial  status 
(e.g.,  dementia,  depression, 

nursing  home -residence)            18/ 

e)  nurse  documentation  of  prior 
and  chronic  disease, 
functional  status,  habits, 

and  psychosocial  status            19/ 


f)  Check  here  if  the  record  demonstrates  evidence  that  the  physician  has 
ready  access  to  additional  records  that  supplement  the  current  data 
regarding  the  patient's  prior  condition. 

Please  rate  the  physician'  initial  assessment  of  acute  medical  problems  present  at 
admission.     Base  your  answer  on  the  history,  physical,   and  labs. 

very 

excellent    good    adequate    poor  poor 
(1)  (2)         (3)         (4)  (5) 

a)  completeness  of  initial  data 

gathering            21/ 


20  / 


b)  integration  of  admission 
information  and  development 

of  appropriate  diagnoses            22/ 


c)  initial  treatment  plan  and 

initial  orders            23 1 
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3)    Assume  adequate  to  optimal  care  and  assume  the  patient  recovers  from  this 

episode  of  acute  myocardial  infarction.     What  do  you  believe  is  this  patient's  life 
expectancy? 

<  1  month    (1)  24/ 

1-6  months    (2) 

>  6  months-1  year    (3) 

>•  1  year    (4) 


4)     Assume  adequate  to  optimal  care  by  the  physician  and  hospital.  How 

effective  is  medical  science  in  treating  this  patient's  acute  illness  or 
in  preventing  worsened  health  status  due  to  the  illness?    Consider  the 
severity  of  the  patient's  acute  illness  and  the  patient's  chronic  reserve. 


very  effective 

effective 

not  so  effective 

very  ineffective 


(1) 
(2) 
(3) 
(4) 


25/ 


5)     Considering  the  entire  hospitalization,  on  average,  was  use  of  these 
services  appropriate  with  respect  to  the  patient's  needs?     If  not 
appropriate,  was  it  because  of  underuse? 

definitely    probably    probably    definitely  under- 
use 


yes 
(1) 


yes 
(2) 


no 

(3) 


no 
(4) 


a)  monitoring  intensity 

ai)     intensive  care 

aii)  telemetry  without 
intensive  care 

b)  respiratory  therapy 
delivered 


c)        and  ventilation 


d)  arterial  blood  gases 

e)  physical  therapy 
delivered 

f)  EKGs 

g)  chest  x-rays 

h)  venous  blood 
tests ,  urinalyses , 
sputum  analyses 

i)  consultations 

j)  medications  (type 
and  route) 


a) 

Qai) 
I — I  aii) 


a, 


26/ 
27/ 

28/ 

29-30/ 
31-32/ 

33-34/ 

35-36/ 
37-38/ 
39-40/ 

41-42/ 
43-44/ 

45-46/ 
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ANSWER  QUESTION  6  ONLY  IF  THE  PATIENT  WAS  DISCHARGED  ALIVE 

6)    Was  length  of  stay  appropriate  given  the  patient's  status  at  discharge 
and  disposition  plans? 

definitely  yes    (1)  47/ 

probably  yes    (2) 

probably  no    (3) 

definitely  no    (4) 

6a)  If  probably  or  definitely  not  appropriate,  how  would  you  describe 
length  of  stay? 

too  short    (1)  48/ 

too  long    (2) 


6b)  If  length  of  stay  was  too  short  or  too  long,  what  were  the  apparent 
reasons?     Check  one  or  more  reasons  if  applicable. 

Too  short  Too  long 

i)  Patient  too  unstable  H  1-  v)  Waiting  for  nursing  49-50/ 

\  1  home  or  ECF  bed 


ii)  Work-up  incomplete  -j  j-         vi)  Waiting  for  home 

-I  1-  care  support  service 

iii)  Rehabilitation  incomplete      -j  1-        vii)  Patient  or  family  H  j-  53-54/ 

-I  1-  refused  discharge  -I  1- 

iv)  Patient  or  family  insisted     I      I       viii)  Waiting  for  procedure  -I  1-  55-56/ 

on  discharge  -I  1-  -I  r 

ANSWER  QUESTION  7  ONLY  IF  THE  PATIENT  DIED  DURING  THE  HOSPITALIZATION 

7)    Was  the  patient's  death  preventable? 

definitely  preventable    (1)  57/ 

probably  preventable    (2) 

probably  not  preventable    (3) 

definitely  not  preventable    (4) 
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ANSWER  THE  REMAINING  QUESTIONS  FOR  ALL  PATIENTS 

8)    How  would  you  characterize  the  patient's  outcome  at  discharge? 

much  better  than  expected    (1)  58/ 

better  than  expected    (2) 

as  expected    (3) 

worse  than  expected    (4) 

much  worse  than  expected    (5) 


9)  Considering  everything  you  know  about  this  patient,  please  rate  overall 
quality  of  care. 

extreme,  above  standard    (1)  59/ 

above  standard    (2) 

adequate    (3) 

below  standard    (4) 

extreme,  below  standard    (5) 

10)  Would  you  send  your  mother  to  these  physicians  in  this  hospital? 

definitely  yes    (1)  60/ 

probably  yes    (2) 

probably  no    (3) 

definitely  no    (4) 


VI.  PPS  AND  IMPAIRMENT  AT  DISCHARGE:  THE 
QUICKER- AND- SICKER  STORY  REVISITED 


Jacqueline  Kosecoff,  Ph.D.,  Katherine  L.  Kahn,  M.D., 
William  H.  Rogers,  Ph.D.,  Ellen  J.  Reinisch,  M.S., 
Marjorie  J.  Sherwood,  M.D.,  Lisa  V.  Rubenstein,  M.D.,  MSPH, 
David  Draper,  Ph.D.,  Carol  P.  Roth,  RN,  MPH, 
Carole  Chew,  RRA,  MPH,  Robert  H.  Brook,  M.D.,  Sc.D. 


Since  the  introduction  in  1983  of  PPS,  there  has  been  considerable  anecdotal  evidence 
that  patients  are  leaving  the  hospital  "quicker  and  sicker"  (Coulton,  1988;  Coe  et  al.,  1986; 
and  Neu  and  Harrison,  1988).  However,  there  has  been  a  dearth  of  systematic  evidence  to 
support  the  assertion  that  post-PPS  patients  are  discharged  sicker.  The  evidence  that  has 
been  available  emanates  largely  from  Congressional  testimony  about  individual  cases  or  from 
nonsystematic  surveys  of  patients  or  physicians. 

We  do  know  that  PPS  has  been  associated  with  dramatic  decreases  in  length  of  stay. 
For  instance,  between  1981  and  1986,  length  of  stay  fell  21  percent  for  congestive  heart 
failure  (2.3  days),  18  percent  for  acute  myocardial  infarction  (2.3  days),  14  percent  for  pneu- 
monia (1.7  days),  28  percent  for  hip  fracture  (5.6  days),  and  32  percent  for  cerebrovascular 
accident  (5.1  days)  (Sec.  VII).  Although  this  trend  for  decreases  in  length  of  stay  has  stabi- 
lized, it  has  not  reversed  (Witsberger  and  Kominsky,  1990).  This  decrease  in  length  of  stay, 
if  associated  with  increased  illness  or  impairment  at  hospital  discharge,  might  adversely 
affect  a  patient's  recovery. 

This  section  provides  nationally  representative  data  about  the  level  of  impairment  at 
discharge  before  and  after  the  implementation  of  PPS.  To  answer  correctly  the  question  of 
whether  patients  are  inappropriately  discharged  sicker  since  the  implementation  of  PPS,  we 
must  first  define  a  valid  measure  of  impairment  at  discharge  and  then  determine  if  patients 
discharged  from  the  hospital  sicker  do  worse  than  expected.  If  patients  who  are  discharged 
clinically  impaired  die  more  frequently  than  expected,  then  allowing  an  increase  in  the 
number  of  patients  discharged  with  an  instability  would  appear  to  be  unwise.  On  the  other 
hand,  if  such  patients  are  discharged  for  recovery  to  nursing  homes  or  to  care-givers  at  home 
who  can  handle  an  increased  level  of  impairment  without  a  concomitant  increase  in  deaths, 
then  "earlier"  and  "sicker"  discharges  would  not  be  an  inappropriate  outcome. 

METHODS 

The  study  methods,  including  design,  sampling,  and  analysis  strategy,  were  described 
in  Sec.  II. 

This  section,  because  it  concerns  patients  discharged  alive  from  the  hospital,  excludes 
patients  who  died  in  the  hospital,  who  were  directly  transferred  to  other  acute  care  hospitals, 
or  who  had  a  DNR  order  written  some  time  during  the  hospitalization.  These  exclusions 
reduced  the  maximum  sample  size  from  14,012  to  10,913.  The  sample  does  include  patients 
who  were  admitted  to  the  hospital  from  a  nursing  home.  The  results  reported  here  are  not 
qualitatively  affected  by  excluding  patients  who  were  in  a  nursing  home  before  hospitaliza- 
tion. (See  App.  E.) 
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CONSTRUCTION  OF  MEASURES  FOR  DISCHARGE  IMPAIRMENT 

Using  clinical  judgment,  we  constructed  three  measures  of  discharge  impairment  (see 
Table  6.1):  instability  at  discharge,  sickness  at  discharge,  and  abnormal  last  laboratory 
values.  Data  used  to  construct  the  first  two  measures  came  primarily  from  the  day  of 
discharge  or  day  before  discharge.  Since  most  patients  are  discharged  in  the  morning,  this 
means  that  the  abnormality  was  almost  always  noted  in  the  last  24  hours  of  hospital  admis- 
sion. 

Instability  variables  are  designed  to  identify  patient  problems  present  at  discharge  (1) 
that  clinicians  generally  agree  should  be  corrected  before  discharge  and  (2)  that  are  likely  to 
cause  poor  outcomes  if  not  corrected.  In  general,  instability  variables  reflect  clinical  prob- 
lems that  first  occurred  before  discharge  and  that  were  not  present  at  admission.  Sickness 
variables  are  designed  to  measure  sickness  at  discharge  regardless  of  whether  the  problem 
should  or  could  have  been  corrected  at  discharge.  The  key  distinction  between  our 
definitions  of  instability  and  sickness  rests  with  the  "correctability"  of  the  conditions.  For 
example,  the  sickness  variables  include  patients  who  have  chest  pain  on  the  day  before  the 
day  of  discharge  even  though  they  may  have  had  chest  pain  daily  for  weeks.  The  instability 
variables,  on  the  other  hand,  include  a  subset  of  the  patients  with  sickness  at  discharge.  In 
this  case,  only  those  who  have  chest  pain  at  discharge  but  not  at  admission  are  included.  As 
seen  in  Table  6.1,  8.3  percent  of  the  AMI  patients  had  sickness  at  discharge  caused  by  the 
presence  of  chest  pain  on  at  least  one  of  the  last  two  days  of  hospitalization,  but  0.3  percent 
of  AMI  patients  had  instability  at  discharge  caused  by  the  presence  of  chest  pain  that  was 
not  present  at  admission. 

Abnormal  last  laboratory  values  reflect  the  presence  of  abnormal  findings  the  last  time 
a  test  was  performed  during  the  hospitalization,  regardless  of  when  the  value  was  obtained. 
The  three  measures  of  impairment  (instability  at  discharge,  sickness  at  discharge,  and 
abnormal  last  laboratory  values)  consisted  of  a  maximum  of  10, 11,  and  8  items,  respectively. 

ANALYSIS 

Virtually  all  the  analyses  reported  in  this  section  are  based  on  cross-tabulations  and 
chi-square  tests  of  significance.  However,  we  did  use  multivariate  techniques  (linear  and 
logistic  regression)  to  adjust  for  differences  in  patient  sickness  at  the  time  of  hospital  admis- 
sion when  studying  discharge  impairment  before  and  after  prospective  payment  (Sec.  III). 
The  results  from  this  analysis  are  qualitatively  consistent  with  the  simpler  unadjusted 
analyses  reported  here  (App.  E).  In  addition,  emphasis  is  given  in  this  section  to  the  instabil- 
ity at  discharge  measure  because  it  is  clinically  the  most  appealing  of  our  three  discharge 
measures.  It  consists  of  symptoms  and  signs,  almost  all  newly  present  at  discharge,  that  we 
believe  are  more  likely  to  be  correctable  or  in  need  of  urgent  attention  than  impairments  that 
have  been  present  throughout  the  course  of  the  hospitalization.  A  patient  with  heart  failure 
who  is  discharged  from  the  hospital  a  little  short  of  breath  when  he  previously  was  severely 
short  of  breath  probably  represents  a  different  and  less  remediable  situation  than  a  patient 
who  is  discharged  short  of  breath  without  having  had  that  symptom  during  the  earlier  part 
of  the  hospitalization.  In  addition,  because  the  abnormal  last  laboratory  measure  potentially 
used  data  that  were  obtained  before  the  immediate  discharge  period,  it  was  felt  to  be  less 
precise  as  a  clinical  measure  of  impairment  at  discharge. 
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Table  6.1 


PERCENTAGE  OF  PATIENTS  DISCHARGED  WITH  IMPAIRMENT,  BY  MEASURES  OF  IMPAIRMENT 


CHF 

AMI 

PNE 

CVA 

HIP 

5  Diseases 

Measures 

(n=2,348) 

(n=  1,946) 

(n=2,141) 

(n=  1,927) 

(n=2,551) 

(n=10,913) 

Instability  at  Discharge4 

Fever  (temperature  >  101°F) 

0.0 

0.2 

0.2 

0.3 

0.2 

0.2 

New  incontinence 

4.6 

5.8 

15.6 

12.3 

9.5 

New  chest  pain 

1.4 

0.3 

1.3 

2.8 

1.5 

New  shortness  of  breath 

1.5 

1.1 

1.5 

1.5 

(b) 

1.4 

New  confusion 

4.5 

2.8 

2.5 

5.5 

3.9 

New  heart  rate  2  130 

0.7 

0.2 

0.5 

0.4 

0.2 

0.4 

New  respiratory  rate  a  30 

1.9 

0.7 

1.8 

1.8 

1.5 

1.6 

Diastolic  BP  s  105 

1.5 

0.5 

1.1 

2.9 

0.6 

1.3 

New  systolic  BP  <  90 

2.1 

1.6 

1.2 

1.4 

0.7 

1.4 

New  low  heart  rate  <  50 

1.8 

New  premature  ventricular  contractions 

0.9 

s  1  instability 

15.4 

9.5 

14.3 

21.3 

21.0 

16.5 

Mean  no.  instabilities 

0.18 

0.10 

0.16 

0.24 

0.24 

0.18 

Mean  no.  instabilities  among  patients  with  2 

1  instability  1.2 

1.1 

1.1 

1.1 

1.1 

1.1 

Sickness  at  Discharge3 

Fever  (temperature  <  101°F) 

0.0 

0.2 

0.2 

0.3 

0.2 

0.2 

New  incontinence 

4.6 

5.8 

15.6 

12.3 

9.5 

Chest  pain 

5.5 

8.3 

4.8 

3.3 

5.3 

Shortness  of  breath 

29.5 

7.8 

15.8 

3.7 

(b) 

15.0 

Confusion 

13.2 

7.6 

11.1 

10.4 

10.7 

Heart  rate  a  130 

1.2 

0.4 

0.8 

0.4 

0.2 

0.6 

Respiratory  rate  a  30 

5.5 

1.3 

8.5 

3.3 

2.4 

4.2 

Diastolic  BP  2  105 

1.5 

0.5 

1.1 

2.9 

0.6 

1.3 

Systolic  BP  <  90 

4.4 

3.6 

1.9 

1.6 

0.9 

2.4 

New  decubitus 

1.2 

2.1 

2.4 

8.6 

3.8 

Low  heart  rate  <  50 

4.1 

Premature  ventricular  contraction 

2.3 

Parenteral  antibiotics 

36.1 

a  1  sickness 

47.6 

29.4 

60.4 

25.2 

30.8 

39.0 

Mean  no.  sicknesses 

0.67 

0.36 

0.88 

0.30 

0.39 

0.52 

Mean  no.  sicknesses  among  patients  with  2  ] 

sickness  1.4 

1.2 

1.5 

1.2 

1.3 

1.3 

Abnormal  Last  Laboratory  Values0 

Potassiumd 

6.3 

3.8 

7.8 

7.1 

6.7 

6.3 

Sodium6 

f 

Renal  distress 

2.6 

2.2 

2.9 

1.8 

2.8 

2.5 

4.1 

2.4 

2.6 

1.9 

1.6 

2.5 

Hematocrit  Qow)g 
WBC  (high)" 

2.8 

0.5 

2.1 

3.9 

2.3 

9.3 

12.7 

11.0 

13.8 

11.7 

Weight  increase  >  3  lb 

2.9 

2.2 

2.6 

CHF  by  X-ray  worsening 

4.3 

0.9 

7.0 

4.2 

4.1 

>  1  laboratory  abnormality 

27.6 

10.7 

29.2 

20.9 

29.1 

24.1 

Mean  no.  abnormalities 

0.32 

0.11 

0.33 

0.24 

0.33 

0.27 

Mean  no.  abnormalities  among  patients  with 

2  1  abnormality  1.16 

1.05 

1.14 

1.14 

1.13 

1.13 

All  measures  of  discharge  instability  and  sickness  evaluate  the  presence  of  symptoms  or  signs  on  the  day  before  discharge  or  the 
day^of  discharge. 

For  hip  fracture,  chest  pain  is  defined  as  chest  pain  or  shortness  of  breath.  Other  entries  marked  with  a  dash  ( — )  indicate  that  this 
variable  was  not  included  in  the  definition  of  impairment  for  a  particular  procedure.  For  example,  new  incontinence  is  not  included  in 
instability  or  sickness  for  AMI. 

cEach  reported  value  is  the  last  inhospital  value.  When  comparisons  are  made,  the  last  value  is  compared  with  the  most  recent  prior 
inhospital  value.  If  only  one  value  is  available  during  the  hospital  stay,  that  value  is  counted  as  the  last  laboratory  value  even  if  it  was 
also^the  first. 

Abnormal  potassium  (meq/1)  is  defined  as  a  value  s  2.9,  a  value  2  6.0,  or  a  value  between  5.5  and  5.9  but  rising  or  3.0-3.4  and  fall- 
ing. 

Abnormal  sodium  (meq/1)  is  defined  as  a  value  <  119,  a  value  2  155,  or  a  value  between  120  and  129  but  falling,  or  between  150 
and ^.54  but  rising. 

Renal  distress  (mg/dl)  is  defined  as  a  creatinine  value  >  6.0,  a  rise  in  creatinine  >  0.5,  a  BUN  value  >  100,  or  a  BUN  rise  2  25. 
j^Low  hematocrit  (%)  is  defined  as  <  24.9  or  between  25-34.9  with  a  drop  a  7.0  (the  drop  was  not  measured  for  pneumonia). 
High  white  blood  cell  count  (cells  x  1,000/cu  mm)  is  defined  as  2  12  and  rising  (the  rise  was  not  measured  for  pneumonia). 
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RESULTS 

Considering  all  years  of  the  study,  both  pre-  and  post-PPS,  about  one  out  of  six  patients 
(17  percent)  were  discharged  with  at  least  one  instability,  two  out  of  five  patients  (39  percent) 
were  discharged  with  at  least  one  measure  of  sickness,  and  about  one  out  of  four  (24  percent) 
had  an  abnormal  last  laboratory  value  (see  Table  6.1).  (See  App.  E.)  The  presence  of  one  or 
more  instabilities  at  discharge  varied  across  diseases  by  a  factor  of  two,  with  about  one  in  ten 
patients  who  had  a  myocardial  infarction  being  discharged  with  one  or  more  instabilities, 
compared  to  one  in  five  patients  with  cerebrovascular  accident  or  hip  fracture.  Among  those 
with  one  or  more  measures  of  instability,  most  had  just  one  impairment,  with  a  mean 
number  of  instabilities  for  those  who  had  one  or  more  instabilities  varying  by  disease  from 
1.1  to  1.2.  Thus,  almost  all  information  about  this  measure  can  be  captured  by  a  dichoto- 
mous  variable,  zero  instabilities  at  discharge  compared  with  one  or  more. 

The  development  of  new  incontinence  of  urine  contributed  most  to  labeling  a  person  as 
unstable  at  discharge  for  the  four  diseases  for  which  it  was  measured  (i.e.,  excluding  acute 
myocardial  infarction).  For  patients  with  acute  myocardial  infarction,  the  most  predominant 
impairment  was  new  confusion.  Confusion  was  also  an  important  contributor  to  instability 
for  patients  with  the  other  four  conditions  as  well. 

The  degree  of  sickness  at  discharge  varied  among  diseases  from  about  one  in  four 
patients  discharged  with  an  impairment  who  were  admitted  with  a  myocardial  infarction  (29 
percent),  cerebrovascular  accident  (25  percent),  or  hip  fracture  (31  percent),  to  three  out  of 
five  patients  admitted  with  pneumonia  (60  percent).  Like  the  instability  measure,  most  peo- 
ple with  one  or  more  measures  of  sickness  had  only  one,  with  a  range  by  disease  from  1.2  to 
1.5.  New  incontinence  contributed  most  to  the  sickness  measure  for  patients  with  a  cerebro- 
vascular accident  and  hip  fracture,  whereas  for  heart  failure  and  pneumonia  it  was  shortness 
of  breath,  and  for  myocardial  infarction  it  was  chest  pain.  Other  important  contributors  were 
confusion  for  heart  failure,  shortness  of  breath  and  confusion  for  acute  myocardial  infarction, 
confusion  and  increased  respiratory  rate  for  pneumonia,  and  confusion  and  new  decubitus  for 
hip  fracture  patients. 

The  presence  of  one  or  more  abnormal  last  laboratory  values  varied  from  a  low  of  one  in 
ten  patients  discharged  with  a  myocardial  infarction  to  about  one  in  three  patients 
discharged  with  pneumonia  or  hip  fracture.  For  the  four  diseases  in  which  it  was  measured 
(i.e.,  excluding  myocardial  infarction),  the  leading  cause  of  abnormal  last  laboratory  studies 
was  a  high  white  blood  cell  count  (>  12,000  cells/cu  mm  and  rising  in  relation  to  the  prior 
study).  An  abnormal  last  potassium  value  (<2.9,  >6.0,  5.5-5.9  and  rising,  or  3.0-3.4  and  fall- 
ing in  relation  to  the  prior  study — measured  in  meq/1)  was  the  leading  abnormality  for  acute 
myocardial  infarction  patients  and  the  second  leading  abnormality  at  discharge  for  patients 
with  the  other  four  diseases.  About  one  out  of  every  16  patients  (6  percent)  across  all  five 
diseases  was  discharged  with  an  abnormal  potassium  result. 

Using  regression  techniques,  we  determined  the  relationship  between  the  instability  at 
discharge  measure  and  our  measures  of  overall  inhospital  process  of  care  and  sickness  at  the 
time  of  hospital  admission.  Both  measures  are  disease-specific  and  strongly  predict  inhospi- 
tal and  posthospital  death  (Sees.  II-IV).  In  general,  instability  was  not  explained  by  these 
measures  (App.  E).  Only  for  congestive  heart  failure  was  severity  at  admission  related  to 
instability  at  discharge  (p  =  0.04).  All  process  relationships  with  instability  were 
insignificant  except  for  congestive  heart  failure  (p  =  0.02),  and  this  relationship  was  in  the 
"wrong"  direction  (i.e.,  better  process  predicted  more  instability).  Further,  the  amount  of 
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variance  explained  in  these  disease-specific  regressions  varied  from  only  0.5  percent  for 
congestive  heart  failure  to  2.5  percent  for  hip  fracture  patients.  Thus,  instability  at 
discharge  is  not  an  alternative  measure  of  inhospital  process,  and  people  within  a  disease 
category  who  are  sicker  at  admission  are  not  more  likely  to  be  discharged  unstable  relative  to 
those  who  are  less  sick  at  admission. 

Patients  with  instability  were  more  likely  to  be  discharged  to  an  institution  (Table  6.2). 
In  addition,  discharge  to  an  institution  was  associated  with  a  significantly  higher  likelihood 
of  discharge  impairment  according  to  all  three  of  our  measures  (Table  6.3).  On  average, 
about  22  percent  of  patients  discharged  to  an  institution  had  one  or  more  instabilities, 
whereas  13  percent  of  those  going  home  are  so  classified  (relative  risk,  1.8).  The  relative  risk 
is  largest  for  hip  fracture  (relative  risk,  2.0)  and  least  for  myocardial  infarction  (relative  risk, 
1.1).  Sickness  at  discharge  is  the  most  common  impairment.  Fifty  percent  of  patients 
discharged  to  an  institution  had  one  or  more  measures  of  sickness,  whereas  34  percent  of 
those  discharged  to  home  were  so  affected  (relative  risk,  1.5).  For  abnormal  last  laboratory 
values,  32  percent  were  discharged  to  an  institution  with  an  abnormality  compared  to  25  per- 
cent discharged  to  home  (relative  risk,  1.3). 


Table  6.2 

PERCENTAGE  OF  PATIENTS  DISCHARGED  TO  AN  INSTITUTION,  BY  IMPAIRMENT  AT  DISCHARGE 


Measures  of 
Impairment 

One  or  More 
Impairments? 

CHF 

AMI 

PNE 

CVA 

HIP 

5  Diseases 

Instability  at  discharge 

No 

12.9 

7.3 

22.1 

35.5 

56.6 

28.5 

Yes 

20.5  ** 

8.4 

35.9  ** 

61.7  ** 

76.0  ** 

41.7  ** 

Sickness  at  discharge 

No 

11.4 

5.7 

19.5 

34.1 

54.1 

25.4 

Yes 

17.1  ** 

11.4** 

27.1  ** 

61.8** 

75.5  ** 

39.0** 

Abnormal  laboratory 

No 

9.8 

6.7 

22.7 

38.5 

58.8 

27.6 

Yes 

25.5  ** 

13.0** 

27.5  * 

51.1  ** 

65.3  * 

40.5  ** 

*  p  <  0.05,  **  p  <  0.01,  based  on  chi-square  test  of  discharge  destination  of  patients  with  one  or  more 
impairments  compared  with  those  having  no  impairments. 


Table  6.3 

PERCENTAGE  OF  PATIENTS  DISCHARGED  TO  HOME  OR  INSTITUTIONALIZED  WITH  ONE  OR 
MORE  MEASURES  OF  IMPAIRMENT  AT  DISCHARGE 

Measures  of  Discharge 


Impairment 

Destination 

CHF 

AMI 

PNE 

CVA 

HIP 

5  Diseases 

Instability  at  discharge 

Home 

14.1  ** 

9.4 

12.0  ** 

14.2  ** 

12.8  ** 

12.5  ** 

Institution 

22.4 

10.8 

21.3 

32.0 

26.3 

22.4 

Sickness  at  discharge 

Home 

46.2  ** 

28.2  ** 

58.0  ** 

16.8  ** 

19.3  ** 

33.7  ** 

Institution 

57.8 

45.3 

68.0 

37.9 

38.3 

49.5 

Abnormal  laboratory 

Home 

43.5  * 

10.0  ** 

27.9  * 

17.7  ** 

25.8  * 

25.0  ** 

Institution 

49.9 

18.7 

33.3 

26.0 

31.3 

31.8 

Sample  size 

Home 

1,966 

1,736 

1,596 

1,121 

998 

7,417 

Institution 

322 

139 

507 

782 

1,543 

3,293 

NOTE:  *  p  <  0.05,  **  p  <  0.01;  chi-square  test  of  level  of  impairment  of  patients  discharged  home  compared 
with  those  discharged  to  an  institution. 
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Tables  6.4  through  6.9  show  detailed  data  for  various  time  intervals  and  destinations. 
Tables  6.4,  6.5,  6.8,  and  6.9  include  patients  regardless  of  discharge  destination  and  Tables 
6.6  and  6.7  include  only  those  patients  with  a  discharge  destination  of  home.  Tables  6.4 
through  6.7  present  mortality  rates  30  and  90  days  posthospital  discharge,  and  Tables  6.8 
and  6.9  present  mortality  rates  30  and  90  days  posthospital  admission.  Each  table  confirms 
the  previous  findings  and  shows  higher  death  rates  for  patients  discharged  with  instability. 
For  example,  Table  6.4  shows  a  30-day  postdischarge  mortality  rate  of  7.9  for  patients 
discharged  with  instability  as  compared  with  4.0  for  patients  discharged  without  instability 
(p  <  0.05).  This  finding  is  significant  for  each  of  the  study  diseases  except  acute  myocardial 
infarction.  Table  6.6  shows  30-day  postadmission  mortality  rates  for  patients  discharged  to 
home.  For  the  five  diseases  combined,  the  mortality  rate  is  5.5  for  patients  discharged  with 
at  least  one  instability  and  2.5  for  patients  discharged  without  instability  (p  <  0.05). 

The  risk  of  death  at  90  days  following  discharge  is  16  percent  for  patients  discharged 
unstable  and  10  percent  if  instability  was  not  present  at  discharge  (relative  risk,  1.6).  The 
highest  condition-  specific  relative  risk  of  death  within  90  days  postdischarge  was  for  hip 
fracture  patients  (2.0)  and  lowest  for  myocardial  infarction  patients  (1.1). 

Each  table  (Tables  6.4  through  6.9)  presents  the  instability-death  relationship  for  all 
patients  (i.e.,  pre-  and  post-PPS  samples  combined),  for  the  pre-PPS  period  sample  alone,  and 
for  the  post-PPS  period  sample  alone.  The  data  show  that  the  strength  of  the  instability- 
death  relationship  is  somewhat  stronger  pre-PPS  than  post-PPS.  A  survival  analysis  of  90 
days  postdischarge  using  a  Cox  model  and  controlling  explicitly  for  length  of  stay  and  sick- 
ness at  admission  is  shown  in  Table  6.10.  In  Table  6.10  we  see  that  a  patient  with  congestive 
heart  failure  discharged  unstable  in  the  pre-PPS  period  is  1.92  times  as  likely  to  die  (in  log 
terms,  i.e.,  the  ratio  of  the  mortality  rate  is  such  that  its  log  is  0.54)  than  is  the  patient 
discharged  stable.  Similarly,  a  patient  with  congestive  heart  failure  discharged  unstable  in 


Table  6.4 

30-DAY  POSTDISCHARGE  MORTALITY  RATES  FOR  PATIENTS  WITH  AND  WITHOUT  INSTABILITY,  BY  PPS  PERIOD1 

(In  percent) 


CHF  AMI  PNE 


No  No  No 

Period      Instability     Instability  Difference  Instability  Instability  Difference  Instability  Instability  Difference 

Pre-PPS         11.6  5.9  +5.7*  6.6  3.8  +2.8  9.5  1.7  +7.8* 

Post-PPS         8.9  6.1  +2.8  7.4  4.8  +2.6  6.9  4.9       \  +2.0 

All  10.0  6.0  +4.0*  7.0  4.3  +2.7  8.1  3.3  +4.8* 

CVA  HIP  5  Diseasesb 


No 

Period      Instability     Instability    Difference  Instability 


Pre-PPS 

7.0 

4.0 

+3.0* 

7.8 

Post-PPS 

9.5 

4.1 

+5.4* 

4.3 

All 

8.5 

4.0 

+4.5* 

5.8 

No  No 
Instability     Difference     Instability     Instability  Difference 


2.5 

+5.3* 

8.5 

3.6 

+4.9* 

2.5 

+1.8 

7.4 

4.5 

+2.9* 

2.5 

+3.3* 

7.9 

4.0 

+3.9* 

NOTE:  *  =  p  <  0.05  (either  direction)  for  comparison  of  mortality  rates  in  patients  with  and  without  instability. 
aSample  excludes  patients  with  inhospital  death,  DNR  orders  (ever  written  during  the  hospital  stay),  and  transfers  to  acute  care 
hospitals  regardless  of  whether  patient  is  discharged  to  home  or  to  an  institution. 

"Relative  risk  of  death  for  pre-PPS  compared  with  post-PPS  patients  in  five  diseases  combined  is  1.69. 
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Table  6.5 

90-DAY  POSTDISCHARGE  MORTALITY  RATES  FOR  PATIENTS  DISCHARGED  WITH  AND  WITHOUT  INSTABILITY, 

BY  PPS  PERIOD8 

(In  percent) 


CHF 

AMI 

PNE 

No 

No 

No 

Period 

Instability 

Instability 

Difference 

Instability 

Instability 

Difference 

Instability 

Instability 

Difference 

Pre-PPS 

23.2 

15.2 

+8.0' 

10.5 

7.7 

+2.8 

15.9 

7.1 

+8.8* 

Post. PPS 

19.9 

13.4 

+6.5* 

9.5 

10.5 

-1.0 

15.7 

10.7 

+5.5 

All 

21.3 

14.3 

+7.0* 

9.9 

9.1 

+0.8 

15.8 

8.9 

+6.9* 

CVA 

HIP 

5  Diseases'3 

No 

No 

No 

Period 

Instability 

Instability 

Difference 

Instability 

Instability 

Difference 

Instability 

Instability 

Difference 

Pre-PPS 

19.1 

8.9 

+  10.2* 

16.0 

6.5 

+9.5* 

16.9 

9.1 

+7.9* 

Post-PPS 

17.7 

10.0 

+7.7* 

10.4 

5.9 

+4.5* 

14.6 

10.1 

+4.6* 

All 

18.3 

9.5 

+8.8* 

12.8 

6.2 

+6.6* 

15.6 

9.6 

+6.0* 

Relative  Riskb  =  1.7 

NOTE:  *  =  p  <  0.05  (either  direction)  for  comparison  of  mortality  rates  in  patients  with  and  without  instability. 
aSample  excludes  patients  with  inhospital  death,  DNR  orders  (ever  written  during  the  hospital  stay),  and  transfers  to 
acute  care  hospitals  regardless  of  whether  patient  is  discharged  to  home  or  to  an  institution. 

^Relative  risk  of  death  for  pre-PPS  compared  with  post-PPS  patients  in  five  diseases  combined. 


the  post-PPS  period  is  31  percent  more  likely  to  die  than  is  the  patient  discharged  stable. 
The  data  suggest  some  change  in  the  relationship  between  instability  and  postdischarge  mor- 
tality across  time  periods.  The  relationship  differs  across  disease  but  is  insignificant  for  each 
disease.  However,  the  t  for  interaction  is  in  the  same  direction  for  each  disease.  For  the  five 
diseases  combined,  the  post-PPS  relationship  between  instability  at  discharge  and  post- 
discharge  mortality  is  significant  (t  =  -2.15).  Possible  reasons  for  this  are  discussed  in  Sec. 
VIII. 

For  example,  Table  6.11  shows  that  unstable  patients  with  congestive  heart  failure  in 
the  pre-PPS  period  were  1.88  times  as  likely  to  die  within  90  days  of  admission  as  stable 
patients.  Table  6.11  shows  a  survival  analysis  to  90  days  postadmission,  using  a  proportional 
hazards  (Cox)  model  in  which  patients  come  into  the  risk  set  when  they  are  discharged  from 
the  hospital. 

Tables  6.5  through  6.11  show  that  instability  at  discharge  results  in  increased  mortality 
in  both  the  pre-PPS  and  post-PPS  periods  for  all  five  study  diseases.  This  effect  appears  to 
be  somewhat  reduced  in  the  post-PPS  period. 

Table  6.12  shows  the  frequency  of  instability  problems  in  the  pre-  and  post-PPS  periods. 
We  call  attention  to  this  here  because  an  argument  could  be  put  forward  that  the  changes  in 
instability  are  due  to  differences  in  styles  of  recording  or  documentation  in  the  post-PPS 
period  compared  with  the  pre-PPS  period.  However,  the  data  do  not  support  this.  When  we 
group  the  variables  into  two  sets — one  recording-sensitive  (symptoms)  and  one  recording- 
insensitive  (vital  signs) — we  find  that  the  recording-insensitive  measures  actually  increased 
at  a  higher  rate  than  the  sensitive  ones.  The  relative  risk  for  instability  at  discharge  post- 
PPS  compared  with  pre-PPS  is  1.3  for  symptoms  and  1.4  for  signs  (Table  6.13).  The  relative 
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Table  6.8 


30-DAY  POSTADMISSION  MORTALITY  RATES  FOR  PATIENTS  DISCHARGED  STABLE  AND  UNSTABLE,  BY  PPS  PERIOD3 

Mortality  rates  30  days  postadmission  (%) 


/""TIC 

L-rir 

AJVil 

p\rc 
rfNc. 

No 

No 

No 

Period 

Instability 

Instability 

Difference 

Instability 

Instability 

Difference 

Instability 

Instability 

Difference 

Pre-PPS 

7.3 

3.2 

+4.1* 

1.3 

1.6 

-0.3 

4.0 

0.7 

+3.3* 

Post-PPS 

5.2 

3.8 

+  1.4 

2.1 

2.5 

-0.4 

5.7 

2.5 

+3.2* 

All 

6.1 

3.5 

+2.6* 

1.8 

2.0 

-0.2 

4.9 

1.6 

+3.3* 

CVA 

HIP 

5  Diseases^ 

No 

No 

No 

Period 

Instability 

Instability 

Difference 

Instability 

Instability 

Difference 

Instability 

Instability 

Difference 

Pre-PPS 

3.2 

0.9 

+2.3* 

3.9 

0.7 

+3.2* 

3.9 

1.4 

+2.5* 

Post-PPS 

6.3 

2.4 

+3.9* 

2.5 

1.5 

+1.0 

4.4 

2.5 

+  1.8* 

All 

5.0 

1.7 

+3.3* 

3.1 

1.1 

+2.0* 

4.2 

2.0 

+2.2* 

Relative  Riskb=  1.4 


NOTE:  *  —  p  <  0.05  (either  direction)  for  comparison  of  mortality  rates  in  patients  with  and  without  instability. 
aSample  excludes  patients  with  inhospital  death,  DNR  orders  (ever  written  during  the  hospital  stay),  and  transfers  to  acute  care  hos- 
pitals regardless  of  whether  patient  is  discharged  to  home  or  to  an  institution. 

''Relative  risk  of  death  for  pre-PPS  compared  with  post -PPS  patients  in  five  diseases  combined. 


Table  6.9 


90-DAY  POSTADMISSION  MORTALITY  RATES  FOR  PATIENTS  DISCHARGED  STABLE  AND  UNSTABLE,  BY  PPS  PERI0D: 

Mortality  rates  90  days  postadmission  (%)a 


CHF 

AMI 

PNE 

No 

No 

No 

Period 

Instability 

Instability 

Difference 

Instability 

Instability 

Difference 

Instability 

Instability 

Difference 

Pre-PPS 

21.0 

12.9 

+8.1* 

10.5 

6.4 

+4.1 

15.0 

6.0 

+9.1* 

Post-PPS 

19.9 

12.7 

+7.2* 

9.5 

9.2 

+0.3 

13.8 

9.4 

+4.4 

All 

20.4 

12.8 

+7.6* 

9.9 

7.8 

+2.1 

14.4 

7.7 

+6.7* 

CVA 

HIP 

5  Diseases'' 

No 

No 

No 

Period 

Instability 

Instability 

Difference 

Instability 

Instability 

Difference 

Instability 

Instability 

Difference 

Pre-PPS 

14.7 

7.4 

+7.3* 

14.1 

4.6 

+9.5* 

15.1 

7.5 

+7.6* 

Post-PPS 

15.8 

9.1 

+6.7* 

9.3 

5.2 

+4.1* 

13.7 

9.1 

+4.5* 

All 

15.3 

8.3 

+7.0* 

11.3 

4.9 

+6.4* 

14.3 

8.3 

+6.0* 

Relative  Riskb  -  1.7 

NOTE:  *  =  p  <  0.05  (either  direction)  for  comparison  of  mortality  rates  in  patients  with  and  without  instability. 
aSample  excludes  patients  with  inhospital  death,  DNR  orders  (ever  written  during  the  hospital  stay),  and  transfers  to  acute  care  hos- 
pitals regardless  of  whether  patient  is  discharged  to  home  or  to  an  institution. 

bRelative  risk  of  death  for  pre-PPS  compared  with  post-PPS  patients  in  five  diseases  combined. 
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Table  6.10 


SURVIVAL  ANALYSIS  FOR  PATIENTS  WITHIN  90  DAYS  POSTDISCHARGE  USING 
COXa  AND  SICKNESS-AT-ADMISSION  CONTROLS15  FOR  LENGTH  OF  STAY0 


Log  Hazardd 

Pre- 

Post- 

tfor 

PPS 

PPS 

Interaction 

Congestive  heart  failure 

0.54 

0.31 

-0.8 

Acute  myocardial  infarction 

0.52 

-0.08 

-1.1 

Pneumonia 

0.62 

0.31 

-0.9 

Cerebrovascular  accident 

0.52 

0.29 

-0.8 

Hip  fracture 

0.63 

0.24 

-1.2 

aCox  model  is  the  proportional  hazards  model  introduced  by  Cox  and  explained  in 
Kalbfieish  and  Prentice  (1980). 

Controls  for  length  of  stay:  The  model  considers  only  postdischarge  information  and 
controls  for  length  of  stay  using  a  regression  equation. 

cAdjusted  for  sickness  at  admission  with  sickness-at-admission  scale  designed  to 
predict  death  at  30  days  postadmission  (see  Sec.  III). 

The  unit  for  expressing  differences  is  the  log  of  the  hazard  rate.  For  example,  for 
congestive  heart  failure  in  the  pre-PPS  period,  unstable  patients  were  exp  (0.54)  or  1.92 
times  as  likely  to  die  within  90  days  of  admission  as  stable  patients. 


Table  6.11 


SURVIVAL  ANALYSIS  FOR  PATIENTS  WITHIN  90  DAYS  POSTADMISSION  USING 
COXa  AND  SEVERITY  CONTROLSb  FOR  LENGTH  OF  STAY0 


Log  Hazard^ 

Pre- 

Post- 

tfor 

PPS 

PPS 

Interaction 

Congestive  heart  failure 

0.63 

0.41 

-0.8 

Acute  myocardial  infarction 

0.65 

0.10 

-1.1 

Pneumonia 

0.75 

0.27 

-1.3 

Cerebrovascular  accident 

0.61 

0.30 

-1.0 

Hip  fracture 

0.86 

0.25 

-1.8 

^e  Cox  model  is  the  proportional  hazards  model  introduced  by  Cox  and  explained  in 
Kalbfieish  and  Prentice  (1980). 

Controls  for  length  of  stay:  The  model  considers  only  postdischarge  information  and 
controls  for  length  of  stay  using  a  regression  equation. 

cAdjusted  for  sickness  at  admission  with  (a)  sickness-at-admission  scale  designed  to 
predict  death  at  30  days  postadmission  and  with  (b)  scale  designed  to  measure  sickness  on 
the  tjiird  hospital  day. 

The  unit  for  expressing  differences  is  the  log  of  the  hazard  ratio.  For  example,  for 
congestive  heart  failure  in  the  pre-PPS  period,  unstable  patients  were  exp  (0.63)  or  1.88 
times  as  likely  to  die  within  90  days  of  admission  as  stable  patients. 


increase  in  the  sensitive  measures  averaged  0.31  (in  logs)  and  the  increase  in  the  insensitive 
measures  averaged  0.49. 

Table  6.14  shows  the  hazard  ratios  and  t-statistics  for  the  differences  in  mortality  rates 
for  stable  patients  pre-PPS  and  post-PPS.  For  example,  the  risk  of  death  for  stable  patients 
post-PPS  is  92  percent  of  what  it  was  pre-PPS.  The  positive  sign  on  the  t-statistics  means 
that  death  rates  for  stable  patients  were  (insignificantly)  higher  post-PPS  compared  with 
pre-PPS. 
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Table  6.13 


SYMPTOMS  AND  SIGNS  OF  INSTABILITY  AT  DISCHARGE 
PRE-  AND  POST-PPS 


Pre 

Absolute  % 
Post  Change3 

Relative  % 
Change13 

Symptoms 

Incontinence 

8.5 

10.4 

+  1.9 

22.4 

Chest  pain 

1.3 

1.7 

+0.4 

30.8 

Shortness  of  breath 

1.1 

1.7 

+0.6 

54.6 

Confusion 

3.3 

4.6 

+  1.3 

39.4 

Mean  of  4 

3.6 

4.6 

+  1.1 

36. 8C 

symptoms 

Signs 

Fever 

0.2 

0.2 

0 

0 

Heart  rate  a  130 

0.1 

0.6 

+0.5 

500.0 

Respiratory  rate  a  30 

1.3 

1.8 

+0.5 

38.5 

Diastolic  BP  a  105 

1.2 

1.3 

+0.1 

8.3 

Systolic  BP  <  90 

1.3 

1.4 

+0.1 

7.7 

Heart  rate  <  50 

1.2 

2.5 

+  1.3 

108.3 

PVC 

0.7 

1.3 

+0.4 

57.1 

Mean  of  7  signs 

0.9 

1.3 

+0.4 

102. 8d 

aAbsolute  percentage  change  is  calculated  as  percent  of 
patients  with  symptom  or  sign  (post-PPS)-(pre-PPS). 

^Relative  percent  change  is  calculated  as  percent  of 
patients  with  symptom  or  sign  (post-PPS)-(pre-PPS)/pre-PPS. 

cRelative  risk  for  symptoms  of  instability  at  discharge 
post-PPS  compared  with  pre-PPS  is  4.6/3.6  =  1.3. 

^Relative  risk  for  signs  of  instability  at  discharge  post-PPS 
compared  with  pre-PPS  is  1.3/0.9  =  1.4. 


Having  established  that  instability  at  discharge  is  a  measure  whose  properties  are  of 
interest  to  clinicians  and  policymakers — it  is  not  just  another  measure  of  severity  at  admis- 
sion; neither  does  it  represent  a  general  measure  of  inhospital  process;  and  it  is  associated 
with  higher  likelihood  of  postdischarge  death — we  can  now  consider  instability  at  discharge 
before  and  after  PPS  (Table  6.15).  Post  PPS,  instability  at  discharge  increased  22  percent. 
Before  implementation  of  PPS,  15  percent  of  people  were  discharged  unstable,  whereas  after- 
ward, 18  percent  were  discharged  unstable.  Most  of  the  increase  in  instability  was  concen- 
trated in  those  people  discharged  to  home.  In  particular,  across  all  conditions,  institutions 
did  not  receive  a  significantly  greater  number  of  unstable  patients  (22  percent  pre-PPS  and 
23  percent  post-PPS — an  insignificant  relative  increase  of  3  percent).  On  the  other  hand,  a 
patient  discharged  to  home  post-PPS  was  43  percent  more  likely  to  be  unstable  than  before 
PPS.  This  finding  is  consistent  across  conditions  for  patients  discharged  to  home,  with  the 
biggest  change  occurring  among  hip  fracture  patients  (9  percent  discharged  to  home  were 
unstable  before  PPS,  17  percent  so  discharged  after  PPS — a  93  percent  change). 

Table  6.16  shows  mean  length  of  stay  for  patients  discharged  with  and  without  instabil- 
ity in  the  pre-  and  post-PPS  periods.  Length  of  stay  is  slightly  longer  for  patients  with  insta- 
bility than  for  those  without  instability.  However,  the  difference  in  length  of  stay  for  stable 
and  unstable  patients  drops  from  1.06  days  pre-PPS  to  0.30  days  post-PPS  suggesting  that 
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Table  6.14 

DIFFERENCES  IN  MORTALITY  RATES  FOR 
STABLE  PATIENTS  PRE-  AND  POST-PPS 


Disease 

Log  Hazard3 

tb 

CHF 

-0.08 

-0.6 

AMI 

0.30 

1.6 

PNE 

0.29 

1.6 

CVA 

0.21 

1.1 

HIP 

-0.07 

-0.4 

aThe  unit  for  expressing  the  difference  is 
the  log  of  the  hazard  rates.  For  example,  for 
stable  congestive  heart  failure  patients,  the 
rate  of  death  post-PPS  is  exp  (-0.08)  or  92  per- 
cent as  high  as  the  rate  of  death  for  stable 
patients  pre-PPS. 

^The  t  shows  an  insignificant  difference 
between  the  mortality  rates  for  stable  patients 
pre-PPS  compared  with  post-PPS. 


hospitals  in  the  post-PPS  period  may  be  paying  less  attention  to  unstable  patients.  Table 
6.17  shows  length  of  stay  effects  for  unstable  compared  with  stable  patients  adjusting  for 
sickness  at  admission  (which  did  not  have  a  large  effect).  Table  6.17  shows  the  same  findings 
as  Table  6.16;  the  negative  sign  on  t  for  interactions  for  four  of  the  five  diseases  again  shows 
that  length  of  stay  is  somewhat  shorter  for  unstable  patients  post-PPS  compared  with  pre- 
PPS. 

We  have  elected  to  focus  on  postdischarge  data  rather  than  postadmission  data  as  the 
outcome  of  interest  in  this  section  because  it  is  simple.  In  addition,  the  variable  of  interest, 
unstable  discharge,  happens  only  at  the  end  of  the  hospitalization.  It  violates  regression 
assumptions  about  correlation  between  the  independent  variable  and  the  error  term  (the  ran- 
domness of  death  or  nondeath  in  the  hospital)  to  analyze  a  postadmission  outcome  that  has 
partially  preceded  the  end  of  the  hospitalization.  As  noted  in  Tables  6.4,  6.5,  6.8,  and  6.9,  it 
makes  little  difference  whether  we  use  postadmission  or  postdischarge  as  the  outcome  mea- 
sure. (Note  that  the  sample  for  the  postadmission  data  (Tables  6.8  and  6.9)  excludes  patients 
with  inhospital  death,  since  the  analysis  would  not  otherwise  make  sense). 

DISCUSSION 

Other  sections  in  this  report  have  demonstrated  that  the  introduction  of  PPS  was  not 
associated  with  a  rise  in  either  short-term  (30-day)  mortality  or  six-month  mortality.  In  light 
of  these  findings,  how  important  is  the  finding  that  instability  at  discharge  has  increased 
since  the  advent  of  PPS?  The  answer  depends  on  one's  vantage  point.  We  will  consider 
briefly  the  perspectives  of  the  patient,  the  family,  the  geriatrician,  the  hospital,  the  nursing 
home,  and  finally  the  policymaker. 

From  the  patient's  and  family's  point  of  view,  it  must  be  comforting  to  know  that  the 
overwhelming  majority  of  patients  (85  percent  post-PPS)  discharged  to  home  leave  the  hospi- 
tal in  a  stable  condition.  The  percentage  of  patients  discharged  to  home  with  an  instability, 
however,  has  increased.   Because  the  major  causes  of  these  instabilities  are  clinically 
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Table  6.15 

RELATIONSHIP  OF  PPS  TO  INSTABILITY  AT  DISCHARGE,  BY  DISCHARGE  DESTINATION3 

(In  percent) 


Unstable  at  Discharge 


Di^chartre 

Absolut?  Chanpp 

Rplntivp  C^Hfl'nP'p 

A-  V     A.  CA  L  I  '         \-/  A 1  CI  1  IKy 

Disease 

Destination15 

Pre-PPS 

Post-PPS 

Pre-Post 

Pre-Post 

CHF 

All  Patients 

13.6 

17.1* 

+  3.5 

+  25 

Home 

1.1.7 

16.3** 

+  4.6 

+  39 

Institution 

23.9 

21.2* 

-  2.7 

-  11 

AMI 

All    Tt     J.  *  i 

All  Patients 

8.5 

10.6 

+  2.1 

+  25 

Home 

8.2 

10.8 

+  2.6 

+  32 

Institution 

11.9 

8.3 

-3.6 

-30 

PNE 

All  Patients 

12.6 

16.0* 

+  3.4 

+  27 

Home 

10.6 

13.4 

+  2.8 

+  27 

Institution 

18.6 

23.4 

+  4.8 

+  26 

CVA 

All  Patients 

19.0 

23.4* 

+  4.4 

+  23 

Home 

12.2 

16.0 

+  3.8 

+  31 

Institution 

30.1 

33.5 

+  3.4 

+  11 

HIP 

All  Patients 

18.8 

23.1** 

+  4.3 

+  23 

Home 

8.8 

17.0** 

+  8.2 

+  93 

Institution 

25.7 

26.9 

+  1.2 

+  4 

5  Diseases 

All  Patients 

15.0 

18.3** 

+  3.3 

+  22 

Home 

10.3 

14.7** 

+  4.4 

+  43 

Institution 

22.0 

22.7 

+0.7 

+3 

NOTE:  *  p  <  0.05,  **  p  <  0.01;  for  chi-square  test  of  pre-post  differences  in  instability  at 
discharge  for  all  patients,  for  those  discharged  home,  and  for  those  discharged  to  an  institution. 

aPatients  with  inhospital  death,  DNR  orders,  or  transfer  to  acute  care  hospitals  are  excluded 
from  this  analysis. 

^Discharge  destination  for  all  patients  means  that  this  line  refers  to  all  sample  patients 
regardless  of  discharge  destination. 


important,  the  amount  of  support  needed  at  home  has  increased.  Being  discharged  to  home 
unstable  with  confusion  or  incontinence  at  best  is  an  inconvenience  for  the  family  and  at 
worst  may  expose  the  patient  to  an  increased  risk  of  death. 

Elderly  patients  being  discharged  unstable  suffer  from  those  conditions  that  geriatri- 
cians have  been  calling  attention  to  for  years,  namely,  incontinence  and  confusion.  For 
example,  taking  pre-  and  post-PPS  together,  10  percent  of  patients  were  discharged  with  new 
incontinence  that  was  not  present  at  admission  and  4  percent  had  new  confusion.  These 
findings  raise  questions  such  as:  Are  increases  in  instability  caused  by  inappropriately  early 
discharges,  too  many  tests  in  a  shortened  hospital  stay,  incorrect  use  of  new  medications,  or 
changes  in  nursing  practice  (e.g.,  fewer  nurses  per  patient  and  less  time  available  to  talk 
with  the  patient  or  monitor  incontinence  or  disorientation)?  The  answers  to  these  questions 
require  new  data,  perhaps  clinical  trials  and  the  development  of  valid  disease-specific 
discharge  guidelines. 

From  a  hospital  or  nursing  home  perspective,  our  results  are  both  encouraging  and 
potentially  disturbing.  Instability  among  patients  who  were  not  assigned  DNR  status  and 
who  were  discharged  to  nursing  homes  has  not  increased  post-PPS.  On  the  other  hand,  the 
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Table  6.16 

MEAN  LENGTH  OF  STAY  FOR  UNSTABLE  AND  STABLE  PATIENTS,  BY  PPS  PERIOD 

Mean  length  of  stay  (days) 


CHF  AMI  PNE 


No        Difference  by  No        Difference  by  No        Difference  by 

Period      Instability    Instability     Instability4     Instability    Instability     Instability3     Instability    Instability  Instability3 


Pre-PPS 

11.0 

11.1 

0 

14.9 

15.0 

-0.1 

13.0 

12.2 

+0.8 

Post-PPS 

8.5 

8.6 

-0.1 

13.3 

12.2 

+1.1 

9.3 

10.2 

-0.9 

All 

9.5 

9.8 

-0.3 

14.0 

13.7 

+0.3 

10.9 

11.2 

-0.3 

Difference 

by  PPS 

Periodb 

2.5 

2.5 

0 

1.6 

2.8 

-1.2 

3.7 

2.0 

1.7 

CVA 

HIP 

5  Diseases 

No 

Difference  by 

No 

Difference  by 

No 

Difference  by 

Period 

Instability 

Instability 

Instability3 

Instability 

Instability 

Instability3 

Instability 

Instability 

Instability3 

Pre-PPS 

20.1 

16.5 

+3.6 

20.9 

19.9 

+1.0 

16.0 

14.9 

+  1.06 

Post-PPS 

12.0 

10.5 

+1.5 

14.2 

14.3 

-0.1 

11.5 

11.2 

+0.30 

All 

15.4 

13.5 

+  1.9 

17.1 

17.1 

0 

13.4 

13.1 

+0.32 

Difference 

by  PPS 

Periodb 

8.1 

6.0 

2.1 

6.7 

5.6 

1.1 

4.5 

2.7 

1.8 

difference  by  instability  is  (LOS  for  unstable  patients)  -  (LOS  for  stable  patients). 
^Difference  by  PPS  period  is  (LOS  for  pre-PPS  patients)  -  (LOS  for  post-PPS  patients). 


Table  6.17 

LENGTH  OF  STAY  EFFECTS  FOR  UNSTABLE 
COMPARED  WITH  STABLE  PATIENTS 


Log  Hazard  Ratioa 


tfor 


Pre 

Post 

Interaction 

CHF 

-0.07 

-0.09 

-0.2 

AMI 

-0.00 

+0.02 

+0.3 

PNE 

-0.05 

-0.10 

-0.8 

CVA 

+0.14 

+0.10 

-0.6 

HIP 

+0.03 

+0.01 

-0.3 

aLog  hazard  ratio  describes  the  relative  like- 
lihood of  discharge  on  a  given  day  after  admis- 
sion. The  numbers  in  the  table  represent  the 
relative  risk  of  discharge  with  instabilities  as 
compared  with  stable  dicharges,  censoring  at 
the  date  of  death.  For  example,  for  congestive 
heart  failure  in  the  pre-PPS  period,  unstable 
patients  were  exp  (-0.07)  or  93  percent  as  likely 
to  be  discharged  on  a  given  day  as  stable 
patients. 
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observation  that  one-quarter  of  nursing  home  patients  are  admitted  with  an  instability  is  a 
disquieting  finding.  Is  the  hospital  discharge  planning  process  adequate  to  handle  the 
smooth  transition  of  these  patients  to  institutions,  and  is  the  nursing  home  reimbursement 
level  sufficient  to  provide  adequate  care  for  them?  In  particular,  do  nursing  home  physicians 
know  of  these  problems,  are  they  noted  in  the  clinical  record  sent  to  the  nursing  home,  and 
do  nursing  home  nurses  and  physicians  develop  an  adequate  strategy  to  address  them? 

These  data  might  justifiably  be  greeted  as  an  additional  burden  placed  at  the  hospital 
administrator's  doorstep.  That  may  be  the  case,  but  because  instability  is  linked  to  subse- 
quent death  and  results  in  decreased  quality  of  life,  this  issue  needs  to  be  addressed.  Learn- 
ing at  the  hospital  level  what  causes  instability  at  discharge  and  how  to  reduce  it  should 
become  a  major  administration  and  medical  staff.  Either  decreasing  instability  at  discharge 
to  zero  or  developing  postdischarge  plans  to  eliminate  its  effects  could  become  part  of  a 
hospital's  continuous  improvement  effort.  It  might  also  receive  emphasis  in  an  accreditation 
program  such  as  that  sponsored  by  the  Joint  Commission  for  Accreditation  of  Health  Care 
Organizations. 

Finally,  from  a  policy  perspective,  these  results  raise  concerns  about  what  information 
is  needed  to  monitor  a  change  in  health  policy  and  how  often  data  on  instability  at  discharge 
should  be  collected.  The  data  in  this  report  are  now  four  years  out  of  date,  and  during  that 
time  hospitals  have  received  less  generous  levels  of  reimbursement.  We  have  demonstrated 
a  problem  that  has  been  apparent  since  at  least  1981:  Patients  discharged  to  home  in  an 
unstable  condition  have  a  greater  than  average  expected  risk  of  dying  and  probably  need 
more  family  support.  In  addition,  we  have  demonstrated  that  the  risk  of  being  discharged 
unstable  has  increased  since  PPS  was  implemented.  Reducing  the  effect  of  instability  may 
be  possible  even  without  increasing  hospital  length  of  stay,  but  this  will  require  examining, 
with  the  aid  of  clinical  data,  the  impact  of  various  policy  options  on  instability's  occurrence 
and  outcome. 


VII.  COMPARING  OUTCOMES  OF  CARE  PRE-  AND  POST-PPS 
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Given  that  the  main  goal  of  the  medical  system  is  to  produce  better  patient  outcomes, 
any  evaluation  of  PPS  must  closely  examine  what  happens  to  patients  as  a  function  of  the 
care  they  receive.  To  evaluate  whether  patient  outcomes  have  changed  after  the  implemen- 
tation of  the  combined  PPS/PRO  system,  we  conducted  an  observational  study  in  which  we 
compared  outcomes  before  and  after  PPS  was  introduced.  It  was  not  enough  to  compare  raw 
outcomes,  however;  we  had  to  account  in  addition  for  differences  pre-  compared  with  post- 
PPS  in  patient  characteristics  at  the  time  of  admission.  This  was  particularly  important 
because  sickness  at  admission,  a  strong  predictor  of  outcomes,  changed  noticeably  during  the 
period  we  studied  (Sec.  III). 

In  this  section  we  report  on  changes  in  length  of  stay;  in  mortality,  adjusted  for  sickness 
at  admission,  both  during  the  hospital  stay  and  within  30  and  180  days  of  admission;  in 
discharge  to  home  or  to  an  institution  following  hospitalization;  in  prolonged  nursing  home 
stay;  and  in  hospital  readmissions.  Of  our  three  mortality  outcomes,  only  inhospital  death  is 
available  from  medical  record  review.  Inhospital  mortality,  however,  provides  a  distorted 
view  of  changes  in  outcomes  over  time,  because  length  of  stay  has  declined  post-PPS,  reduc- 
ing the  period  of  observation  for  inhospital  deaths  (Jencks  et  al.,  1988).  We  have  therefore 
studied  both  inhospital  mortality  and  mortality  at  fixed  times  after  hospital  admission  (30 
and  180  days).  We  have  included  outcomes  other  than  mortality  to  provide  a  more 
comprehensive  look  at  what  happened  to  patients. 

METHODS 

Measuring  Sickness  at  Admission 

To  take  account  of  patients'  sickness  at  admission,  we  collected  60-80  disease-specific 
variables  from  the  medical  record  that  described  the  patient's  acute  and  chronic  morbid  and 
comorbid  conditions  (Kahn  et  al.,  1988b;  Rubenstein  et  al.,  1988;  Kosecoff  et  al,  1988;  Sher- 
wood et  al.,  1988;  and  Roth  et  al.,  1988).  Using  these  data  we  developed  disease-specific 
sickness-at-admission  scales  and  used  them  to  adjust  outcomes  for  patient  sickness  at  admis- 
sion. Details  on  how  this  was  done  are  provided  in  Sec.  III. 

Measuring  Mortality 

We  used  the  medical  record  as  our  source  of  inhospital  mortality  information,  and 
already  existing  HCFA  files  to  determine  mortality  status  after  the  patient's  discharge.  By 
using  the  patient's  last  name,  first  name,  date  of  birth,  and  Health  Insurance  Claim  (HIC) 
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number  from  the  medical  record,  we  were  able  to  accurately  match  92  percent  of  the  patients 
in  our  study  sample  to  the  HCFA  Health  Insurance  Master  (HIM)  file. 

We  tried  to  ascertain  whether  the  8  percent  of  patients  we  could  not  match  were  dif- 
ferent in  important  ways  from  the  other  92  percent  of  patients.  Using  the  medical  record 
data,  61  percent  of  the  unmatched  patients  were  from  the  pre-PPS  period  and  39  percent 
were  from  the  post-PPS  period.  In  the  pre-PPS  period  20  percent  of  unmatched  patients  died 
in  the  hospital,  whereas  in  the  post-PPS  period  19  percent  had  died  inhospital.  The 
unmatched  patients  were  sicker  on  average  than  were  matched  patients,  but  the  distribu- 
tions of  sickness  at  admission  among  the  unmatched  patients  were  similar  in  the  pre-  and 
post-PPS  periods,  so  that  any  effects  arising  from  this  disparity  will  approximately  cancel  in 
the  comparisons.  (See  App.  F  for  further  details  about  using  mortality  data  from  the  HIM 
file.) 

We  assessed  short-term  mortality  by  studying  both  inhospital  mortality  and  death 
within  30  days  of  the  acute  care  admission.  To  determine  an  appropriate  measure  of 
medium-term  mortality,  we  regressed  death  from  30  to  90  days,  91  to  180  days,  181  to  270 
days,  and  271  to  365  days  on  process,  sickness  at  admission,  and  instability  at  discharge. 
Sickness  at  admission  was  an  important  predictor  for  all  four  death  period  outcomes,  but  pro- 
cess and  instability  at  discharge  were  only  significant  up  to  180  days.  This  indicates  that 
differences  in  effects  of  care  on  death  during  an  acute  hospitalization  may  attenuate  after 
180  days.  We  therefore  chose  death  within  180  days  postadmission  as  our  indicator  of 
medium-term  mortality. 

Measuring  Preadmission  Residence  and  Discharge  Destination 

We  used  the  medical  record  as  the  source  of  both  the  patient's  preadmission  residence 
and  discharge  destination,  labeling  origin  or  destination  "home"  if  so  indicated  in  the  physi- 
cians', nurses',  or  therapists'  notes  or  in  administrative  data.  For  both  origin  and  destination 
we  defined  "nursing  home"  as  a  skilled  nursing  facility,  intermediate  care  facility,  extended 
care  facility,  or  a  nursing  home  without  other  description.  We  listed  retirement  homes,  shel- 
tered housing,  congregate  housing,  halfway  housing,  and  board  and  care  facilities  separately. 

Measuring  Prolonged  Nursing  Home  Stay 

The  only  available  data  source  for  studying  duration  of  stay  in  a  nursing  home  is 
Medicare's  Part  B  files  of  physician  bills.  When  a  physician  bills  for  a  nursing  home  visit, 
either  the  place  of  service  is  designated  as  a  nursing  home  or  a  special  procedure  code  is 
used.  We  were  able  to  collect  these  kinds  of  data  for  patients  in  three  of  the  five  study  states. 

In  States  A  and  B,  we  report  only  the  number  of  patients  for  whom  a  bill  was  submitted, 
during  months  5,  6,  or  7  after  admission,  for  a  visit  to  a  skilled  nursing  home  or  other 
(residential)  nursing  home,  without  differentiating  between  the  two.  In  State  C  we  report 
the  number  of  patients  for  whom  a  bill  was  submitted  for  a  visit  to  a  skilled  nursing  facility 
only.  We  report  the  data  for  the  states  separately,  because  of  differences  in  the  way  state- 
specific  carriers  categorized  physician  bills,  and  we  then  give  average  values  for  the  three 
states  combined. 

To  investigate  the  validity  of  this  method  for  studying  prolonged  nursing  home  stays, 
we  determined  how  often  patients  whose  inhospital  medical  records  indicated  that  they  were 
discharged  to  a  nursing  home  had  any  physician  bill  from  a  nursing  home  during  the  three 
months  following  discharge.  In  the  three  study  states,  we  found  that  90  percent,  86  percent, 
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and  84  percent,  respectively,  of  patients  discharged  to  a  nursing  home  had  evidence  for  such 
a  physician  bill.  We  then  determined  the  number  of  patients  who  were  alive  seven  months 
after  the  acute  care  hospital  admission  and  who  also  had  a  bill  submitted  by  a  physician  for  a 
visit  to  a  nursing  home  within  five,  six,  or  seven  months  after  hospital  admission. 

Measuring  Readmissions 

To  study  readmissions  we  used  the  patients  we  matched  to  the  HIM  file  (previously 
described  under  postdischarge  mortality)  and  matched  them  to  HCFA's  Bill  Retrieval  file.  Of 
the  92  percent  of  patients  for  whom  we  had  accurate  mortality  data,  we  were  able  to  accu- 
rately match  95  percent  in  the  Bill  Retrieval  file,  for  an  overall  readmission  matching  success 
rate  of  87  percent.  Specifically,  we  were  unable  to  find  any  bill  for  4.4  percent  of  patients  we 
studied  and  we  were  unable  to  find  the  study  hospitalization  bill,  but  could  find  subsequent 
hospital  bills,  for  1  percent  of  the  sample.  The  resulting  file  allowed  us  to  measure  both  the 
incidence  of  a  readmission  within  180  and  365  days  postadmission  and  the  length  of  stay  for 
all  acute  care  hospitalizations.  We  included  readmissions  to  all  acute  care  hospitals  regard- 
less of  the  reason  for  readmission  or  the  relationship  between  the  readmission  and  the  initial 
hospitalization. 

Adjusting  Outcomes  for  Sickness  at  Admission 

For  all  comparisons  of  outcomes  before  and  after  PPS  except  as  noted  below,  we  used 
the  30-  and  180-day  sickness-at-admission  scales  to  adjust  for  differences  in  sickness  at  the 
time  of  hospital  admission  (Sec.  III).  For  length  of  stay,  unadjusted  and  adjusted  results  are 
similar,  and  we  present  unadjusted  data.  For  30-day  postadmission  mortality,  we  adjusted 
pre-  compared  with  post-PPS  differences  using  the  disease-specific  scales  designed  to  predict 
death  within  30  days  of  admission.  For  180-day  postadmission  mortality,  prolonged  nursing 
home  stay,  and  hospital  readmissions,  we  adjusted  using  the  scales  designed  to  predict  death 
as  of  both  30  and  180  days  after  admission.  In  all  cases  where  adjustment  was  performed, 
our  method  of  adjustment  was  regression;  for  example,  to  compute  pre-  and  post-PPS  30-day 
mortality  adjusted  for  sickness  at  admission,  we  regressed  30-day  mortality  on  a  pre-  post- 
PPS  indicator  variable  and  the  30-day  sickness  scales,  and  computed  adjusted  mortality 
rates  pre-  and  post-PPS  from  this  regression.  For  discharge  destination,  we  adjusted  out- 
comes using  both  the  30-day  and  180-day  sickness-at-admission  scales  reported  in  Sec.  Ill  to 
predict  mortality  and  special  scales  to  predict  discharge  to  a  skilled  nursing  facility.  (See 
App.  F  for  the  description  of  scales  to  predict  discharge  to  a  skilled  nursing  facility.)  Neither 
affected  the  comparisons  of  discharge  destination  pre-  and  post-PPS,  and  we  present  the 
discharge  destination  results  without  adjustment  for  sickness  at  admission. 

RESULTS 
Length  of  Stay 

For  each  disease  studied  we  found  a  substantial  reduction  in  length  of  stay  (see  Table 
7.1).  The  reduction  was  21  percent  for  congestive  heart  failure,  18  percent  for  acute  myocar- 
dial infarction,  14  percent  for  pneumonia,  32  percent  for  cerebrovascular  accident,  and  28 
percent  for  hip  fracture.  Overall,  across  the  five  diseases,  we  found  a  24  percent  reduction 
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Table  7.1 

MEAN  LENGTH  OF  STAY  IN  DAYSa,b  BEFORE  AND  AFTER  PPS,  BY  DISEASE 


Disease 

n 

Pre-PPS 

Post-PPS 

Difference 

CHF 

2,824 

11.1 

8.8 

-2.3* 

AMI 

2,853 

12.7 

10.4 

-2.3* 

PNE 

2,749 

12.1 

10.4 

-1.7* 

CVA 

2,824 

16.2 

11.1 

-5.1* 

HIP 

2,762 

20.1 

14.5 

-5.6* 

5  diseases 

14,012 

14.4 

11.0 

-3.4* 

NOTE:  *  p  <  0.001  for  comparison  of  length  of  stay  pre-PPS  compared  with 
post-PPS. 

aUnadjusted  for  sickness  at  admission.  Such  adjustment  had  little  effect  (see  the 
text). 

If  patient  was  discharged  to  a  nonacute  or  swing  hospital  bed,  patient  was  con- 
sidered discharged. 

(3.4  days)  in  length  of  stay.  For  patients  discharged  alive,  length  of  stay  fell  26  percent,  and 
for  patients  with  inhospital  death,  length  of  stay  fell  15  percent  post-PPS  compared  to  pre- 
PPS  for  the  five  diseases  combined.  (Appendix  F  presents  data  for  mean  length  of  stay  pre- 
and  post-PPS  after  adjustment  for  sickness  at  admission.) 

Mortality 

After  adjustment  for  sickness  at  the  time  of  hospital  admission,  we  found  a  reduction  in 
inhospital  mortality  after  the  introduction  of  prospective  payment  for  all  five  diseases. 
Adjusted  inhospital  mortality  for  the  five  diseases  combined  dropped  from  16.1  percent  to 
12.8  percent  (Table  7.2).  The  3.3  percentage  point  reduction  in  inhospital  mortality  (95  per- 
cent confidence  interval,  2.3  percent  to  4.3  percent)  was  significant  (p  <  0.01)  as  was  the 
reduction  in  mortality  for  four  of  the  five  diseases. 

When  we  studied  mortality  in  a  fixed  time  period  after  hospital  admission,  adjusted 
mortality  rates  for  all  five  diseases  remained  lower  post-PPS  compared  with  pre-PPS.  How- 
ever, the  relationship  was  no  longer  statistically  significant  at  the  0.01  level.  Unweighted 
adjusted  mortality  rates  30  days  after  admission  for  the  five  diseases  combined  were  16.7 
percent  pre-PPS  and  15.7  percent  post-PPS,  a  difference  of  1.0  percentage  points  (95  percent 
confidence  interval,  -0.1  percent  to  2.1  percent,  p  =  0.07).  After  reweighting  the  sample  to 
represent  the  nation,  the  30-day  postadmission  adjusted  mortality  rates  were  slightly  dif- 
ferent from  the  unweighted  values  (16.5  percent  pre-PPS  compared  with  15.4  percent  post- 
PPS,  difference  1.1  percentage  points,  p  =  0.04). 

As  of  180  days  postadmission,  the  adjusted  mortality  rates  for  the  five  diseases  com- 
bined were  29.6  percent  pre-PPS  and  29.2  percent  post-PPS.  Thus,  almost  one-third  of  Medi- 
care patients  hospitalized  with  the  five  study  diseases  have  expired  within  six  months  after 
admission.  For  congestive  heart  failure,  cerebrovascular  accident,  and  hip  fracture  mortality 
dropped  (significantly  for  hip  fracture:  17.8  percent  pre-PPS,  14.8  percent  post-PPS,  p  < 
0.05);  for  acute  myocardial  infarction  and  pneumonia  mortality  rose  post-PPS.  Figure  7.1 
shows  survival  curves  adjusted  for  sickness  at  admission  for  each  of  the  five  study  diseases. 
The  pre-  and  post-PPS  curves  are  similar  for  all  conditions. 
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Fig.  7.1 — Survival  curves  comparing  mortality  following  hospital  admission  through 

180  days  pre-  and  post-PPS  for  five  diseases 
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For  all  five  diseases  we  found  that  patients  who  were  admitted  from  home  but 
discharged  to  an  institution  (e.g.,  a  nursing  home)  had  a  higher  adjusted  mortality  rate  than 
did  patients  discharged  to  home.  For  example,  180-day  postadmission  mortality  for  the  five 
diseases  combined,  adjusted  for  sickness  at  admission,  for  patients  coming  from  home  and 
discharged  alive,  was  14  percent  for  patients  discharged  home  and  23  percent  for  patients 
transferred  to  an  institution.  The  adjusted  180-day  postadmission  mortality  rate  for  patients 
living  in  a  skilled  nursing  facility  before  and  after  the  initial  study  hospitalization  was  33 
percent.  Although  mortality  rates  differ  significantly  (p  <  0.05),  comparisons  of  mortality 
rates  pre-  and  post-PPS  after  controlling  for  discharge  destination  do  not  show  significant 
differences.  (Appendix  F  presents  additional  mortality  data  pre-PPS  compared  with  post- 
PPS.) 

Discharge  Destination 

For  the  five  diseases  combined  the  fraction  of  patients  with  a  preadmission  residence  of 
home  and  a  discharge  destination  of  home  was  77  percent  pre-PPS  and  73  percent  post-PPS 
(p  <  0.05),  with  the  most  important  difference  being  for  hip  fracture  patients  (56  percent  pre- 
PPS  and  48  percent  post-PPS,  p  <  0.05,  Table  7.3).  We  found  little  change  in  the  discharge 
destination  for  patients  admitted  from  a  nursing  home.  Overall,  95  percent  of  patients 
admitted  from  a  nursing  home  returned  to  a  nursing  home,  and  this  did  not  vary 
significantly  by  disease  or  time  period.  (Appendix  F  shows  the  frequency  with  which  patients 
with  a  preadmission  residence  of  home  or  nursing  home  have  various  discharge  destinations 
in  the  pre-  compared  with  the  post-PPS  periods  with  and  without  adjustment  for  sickness  at 
admission.) 

Prolonged  Nursing  Home  Stay 

In  studying  prolonged  nursing  home  stay,  we  focused  on  patients  whose  preadmission 
residence  was  home  and  who  were  still  alive  seven  months  after  the  study  hospitalization. 
For  the  five  diseases  combined,  8  percent  of  such  patients  in  States  A  and  B  were  in  some 
type  of  a  nursing  home  approximately  six  months  after  the  acute  hospitalization,  whereas  2 
percent  of  such  patients  in  State  C  were  in  a  skilled  nursing  home  (Table  7.4). 

In  all  three  states,  somewhat  more  patients  had  prolonged  nursing  home  stays  during 
the  post-PPS  period.  Across  the  three  states  we  found  that  5.6  percent  of  patients  in  the 
pre-PPS  period  and  6.6  percent  of  patients  in  the  post-PPS  period  had  prolonged  nursing 
home  stays  (p  >  0.05).  This  finding  was  not  statistically  significant  in  any  state.  (Appendix  F 
presents  more  details  about  the  methods  and  results  for  the  study  of  prolonged  nursing  home 
stays.) 

Readmissions 

We  found  that  the  number  of  patients  who  died  within  180  days  of  admission  for  the  five 
diseases  combined  or  who  had  at  least  one  readmission  was  unchanged  during  the  two  time 
periods:  57  percent  pre-PPS  and  56  percent  post-PPS  (p  >  0.05,  Table  7.5).  Results  varied 
slightly  by  disease,  with  fewer  congestive  heart  failure,  pneumonia,  and  hip  fracture  patients 
post-PPS  having  either  a  death  or  a  readmission  (p  >  0.05)  but  more  patients  with  acute  myo- 
cardial infarction  suffering  one  of  these  two  outcomes  post-PPS  (p  <  0.05). 
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Table  7.4 

PERCENTAGE  OF  PATIENTS  IN  NURSING  HOMES  SIX  MONTHS  FOLLOWING  HOSPITAL 
ADMISSION  AND  ALIVE  SEVEN  MONTHS  AFTER  HOSPITALIZATION, 
BY  STATE  AND  DISEASE  PRE-  AND  POST-PPS3 


State  Ab  State  Bb  State  Cb 


Disease 

c 

n 

Pre 

Post 

nc 

Pre 

Post 

c 

n 

Pre 

Post 

Congestive  heart  failure 

269 

5.1 

4.6 

260 

5.0 

5.0 

260 

0.8 

2.2 

Acute  myocardial  infarction 

280 

1.4 

2.9 

267 

1.6 

0.0 

269 

0.0 

2.0 

Pneumonia 

240 

4.8 

7.8 

236 

3.4 

4.3 

259 

1.7 

2.2 

Cerebrovascular  accident 

275 

10.9 

18.1 

265 

19.0 

18.1 

250 

6.0 

4.5 

Hip  fracture 

302 

10.7 

12.4 

263 

10.7 

12.0 

271 

2.6 

1.9 

5  diseases 

1,366 

6.7 

9.2 

1,291 

7.9 

8.1 

1,309 

2.2 

2.5 

aThe  entries  in  this  table  represent  the  number  of  patients  with  evidence  for  physician  nursing 
home  bills  submitted  within  five,  six,  or  seven  months  after  admission  divided  by  the  number  of 
patients  alive  seven  months  after  admission.  This  analysis  includes  only  patients  with  preadmis- 
sion residence  of  home.  None  of  the  differences  were  significant  at  p  <  0.05. 

Each  fiscal  carrier  providing  data  for  this  analysis  offered  the  data  in  a  different  format.  For 
States  A  and  B,  the  table  compares  the  percentages  of  patients  with  evidence  for  prolonged  nursing 
home  stay,  indicated  by  a  physician  bill,  to  a  skilled  nursing  or  other  (residential)  nursing  home. 
For  State  C,  the  table  compares  the  percentages  with  evidence  for  prolonged  nursing  home  stay,  as 
indicated  by  a  physician  bill,  to  a  skilled  nursing  facility  only. 
.    Pre-  and  post-PPS  sample  combined. 

As  of  365  days  postadmission,  the  proportion  of  those  patients  discharged  alive  who  had 
at  least  one  hospital  readmission  was  lower  post-PPS  for  all  diseases  except  acute  myocardial 
infarction,  and  lower  for  the  five  diseases  combined  (p  <  0.05  for  congestive  heart  failure,  hip 
fracture,  and  overall).  Across  all  diseases  except  acute  myocardial  infarction,  the  total 
number  of  days  spent  inhospital  within  one  year  of  the  study  hospitalization  was 
significantly  lower  post-PPS  than  pre-PPS  (p  <  0.05  for  congestive  heart  failure,  pneumonia, 
hip  fracture,  and  the  five  diseases  combined). 

Summarizing  Comparisons  of  Outcomes  Pre-  and  Post-PPS 

Table  7.6  summarizes  our  outcome  findings.  For  the  five  diseases  combined,  inhospital 
mortality  was  3  percentage  points  lower  post-PPS  compared  with  pre-PPS  (p  <  0.01).  How- 
ever, this  post-PPS  improvement  in  mortality  decreased  to  1.1  percentage  points  by  30  days 
postadmission  and  was  gone  by  180  days  postadmission. 

For  patients  admitted  to  the  hospital  from  home,  for  the  five  diseases  combined,  we 
found  that  4  percent  more  patients  post-PPS  were  not  discharged  to  home  (p  <  0.05).  We 
found  that  an  additional  1  percent  (0-3  percent  depending  on  the  state)  of  patients  with  a 
preadmission  residence  of  home  had  evidence  for  a  prolonged  nursing  home  stay  (p  >  0.05). 
For  the  five  diseases  combined,  the  proportion  of  patients  with  one  or  more  readmissions 
within  one  year  of  the  initial  hospitalization  was  3  percentage  points  lower  post-PPS  than 
pre-PPS  (p  <  0.05). 
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Table  7.6 

SUMMARY  OF  PERCENTAGE  CHANGES  IN  OUTCOMES  POST-PPS  COMPARED  WITH  PRE-PPS 


Outcome 

CHF 

AMI 

PNE 

CVA 

HIP 

5  Diseases 

Inhospital  mortality3 

-3* 

-2* 

-3* 

-5* 

-2* 

-3* 

30-day  postadmission  mortality3 

-2 

0 

0 

-1 

-1 

-1* 

180-day  postadmission  mortality3 

-2 

+1 

+1 

-1 

-3 

-1 

Home  patients  not  discharged  to  home15 

+2 

+4* 

+4* 

+2 

+8* 

+4* 

Prolonged  nursing  home  stay0 

0 

+1 

+2 

+2 

+1 

+1 

180-day  postadmission  mortality  or  readmission  a'd 

-3 

+4* 

-2 

0 

-3 

-1 

365-day  postadmission  readmissiona'e 

-4* 

+4 

-4 

-0 

-6* 

-2 

NOTE:  *  p  <  0.05. 

aAdjusted  for  sickness  at  admission. 

includes  only  patients  admitted  to  hospital  from  home. 

cProlonged  nursing  home  stay  is  measured  by  physician  bill  to  a  skilled  nursing  facility  or  other 
(residential)  nursing  home  within  five,  six,  and  seven  months  postadmission  for  three  states  combined, 
but  only  for  patients  admitted  to  hospital  from  home  and  alive  as  of  seven  months  postadmission. 

dDeath  or  readmission  within  180  days  postadmission  among  all  sampled  patients. 

eReadmission  within  one  year  postadmission  among  all  sampled  patients. 

DISCUSSION 

Before  discussing  the  relationship  between  the  introduction  of  prospective  payment  and 
changes  in  medical  outcomes,  it  is  important  to  place  these  results  in  context  by  noting  the 
severe  burden  of  illness  carried  by  elderly  patients  hospitalized  with  one  of  these  five  condi- 
tions. By  30-days  posthospital  admission,  16  percent  of  these  patients  have  expired,  with  the 
death  rate  climbing  to  29  percent  within  six  months  of  admission.  For  the  surgical 
condition — hip  fracture — the  six-month  mortality  rate  is  only  16  percent,  but  for  the  four 
medical  conditions  it  is  over  33  percent. 

This  burden  of  illness  is  also  shown  by  readmissions.  Of  patients  who  survived  the  ini- 
tial hospitalization,  more  than  half  were  readmitted  in  the  next  year.  This  fraction  is  highest 
for  patients  initially  hospitalized  with  congestive  heart  failure  (66  percent)  and  lowest  for  hip 
fracture  patients  (44  percent). 

In  addition,  25  percent  of  patients  admitted  from  home  and  discharged  alive  are 
discharged  to  an  institution.  Forty-one  percent  of  cerebrovascular  accident  patients  and  52 
percent  of  hip  fracture  patients  previously  living  at  home  are  discharged  to  an  institution. 
These  patients  have  a  higher  risk  of  dying  but  of  the  patients  still  alive  seven  months  after 
hospital  admission  only  6  percent  are  in  a  nursing  home.  Thus,  most  patients  discharged  to 
a  nursing  home  are  there  for  a  relatively  short  stay. 

Before  and  after  the  implementation  of  prospective  payment,  clinicians,  patients,  and 
families  have  feared,  and  in  some  instances  have  reported,  disasters  in  outcomes  of  care  that 
were  thought  to  be  related  to  the  new  financial  incentives.  We  have  measured  outcomes  pre- 
and  post-PPS  on  a  nationally  representative  database  of  more  than  14,000  patients  who  were 
hospitalized  with  one  of  five  diseases  that  make  up  18  percent  of  Medicare  admissions  and  32 
percent  of  deaths  within  30  days.  In  spite  of  these  fears  and  anecdotal  reports,  we  find  no 
significant  changes  for  the  worse  in  either  mortality  at  30  and  180  days  posthospital  admis- 
sion on  the  one  hand  or  readmission  and  prolonged  nursing  home  stay  on  the  other. 

We  did  find  a  significant  increase  in  the  fraction  of  patients  discharged  directly  to  an 
institution,  but  this  does  not  appear  to  have  resulted  in  a  significant  increase  in  prolonged 
nursing  home  stay.  Our  results,  based  on  data  from  about  150  hospitals  in  three  states,  are 
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not  consistent  with  Fitzgerald  et  al.'s  findings  (1987,  1988)  of  large  increases  in  discharges  to 
nursing  homes  and  prolonged  nursing  home  stays  for  hip  fracture  patients  from  one  large 
hospital.  Our  findings  are  consistent,  however,  with  the  clinical  study  of  Mayer-Oaks  et  al. 
(1988)  examining  outcomes  of  intensive  care  unit  patients  in  three  hospitals,  the  study  of 
DesHarnais  et  al.  (1988)  using  secondary  datasets  to  analyze  inpatient  mortality,  and  the 
studies  of  Palmer  et  al.  (1989)  and  Gerety  et  al.  (1989)  that  examined  outcomes  for  hip  frac- 
ture patients. 

The  findings  reported  here  do  not  prove  that  prospective  payment  is  continuing  not  to 
have  a  negative  impact  on  outcome.  Because  of  the  manner  in  which  PPS  was  introduced,  it 
was  not  possible  to  do  a  controlled  trial  of  its  effects,  which  would  have  permitted  comparison 
of  actual  outcomes  in  the  late  1980s  under  PPS  with  outcomes  during  the  same  time  period 
in  the  absence  of  PPS.  Valid  causal  conclusions  from  the  only  available  data,  which  are 
observational  in  character,  are  difficult  to  achieve;  we  explore  these  issues  in  Sees.  V  and 
VIII. 

Moreover,  our  post-PPS  data  came  from  1985-1986,  a  period  in  which  the  financial 
bases  for  prospective  payment  were  still  changing.  The  rates  paid  to  hospitals  at  that  time 
still  included  a  component  based  on  historical  charges  during  the  cost-plus  reimbursement 
era  before  PPS.  In  addition,  the  amount  of  reimbursement  for  a  patient  with  a  given  diag- 
nosis has  been  changing  since  1986,  and  hospitals'  financial  margins  have  diminished.  Fur- 
ther cuts  in  length  of  stay  may  be  harder  to  achieve  without  negative  impacts,  and  evidence 
from  Sec.  VI  suggests  that  clinical  instability  at  discharge  has  increased  post-PPS.  Last, 
although  on  average  outcomes  have  not  worsened,  some  groups  of  patients  may  have  suf- 
fered. Analyses  to  address  this  last  issue  are  ongoing. 

In  sum,  these  data  suggest  that  prospective  payment  was  implemented  in  a  manner 
that  did  not  adversely  affect  overall  outcomes  for  hospitalized  Medicare  patients.  However, 
the  evidence  also  suggests  that  continuous  clinical  monitoring  on  a  national  level  is  indicated 
to  insure  that  the  continuing  changes  in  prospective  payment  do  not  negatively  affect  the 
outcomes  of  care. 


VIII.  QUALITY  OF  CARE  BEFORE  AND  AFTER  PPS: 
A  SUMMARY  OF  EFFECTS 


William  H.  Rogers,  Ph.D.,  David  Draper,  Ph.D.,  Katherine  L.  Kahn,  M.D., 
Emmett  B.  Keeler,  Ph.D.,  Lisa  V.  Rubenstein,  M.D.,  MSPH, 
Jacqueline  Kosecoff,  Ph.D.,  Robert  H.  Brook,  M.D.,  Sc.D. 


Our  study  had  two  purposes:  to  evaluate  the  quality  of  care  given  to  the  nation's  Medi- 
care patients  in  the  pre-  and  post-PPS  periods,  and  to  compare  observed  post-PPS  quality 
with  predictions  of  what  quality  of  care  might  have  been  in  the  post-PPS  period,  in  the 
absence  of  the  PPS  intervention.  In  previous  sections,  we  documented  significant  differences 
in  sickness  at  admission,  processes  of  care,  instability  at  discharge,  and  outcomes  for  Medi- 
care patients  hospitalized  before  and  after  implementation  of  PPS.  Specifically,  we  demon- 
strated that,  during  the  post-PPS  period,  sickness  at  admission  was  higher,  inhospital 
processes  of  care  were  better,  instability  at  discharge  was  higher,  and  mortality  both  30  days 
and  180  days  following  admission  was  lower  or  unchanged,  when  compared  with  the  pre-PPS 
period.  In  this  section  we  attempt  to  sort  out  which  of  these  differences  may  have  been 
caused  by  PPS  and  which  by  other  factors  that  also  changed  during  the  same  time  period. 

Many  changes  with  the  potential  to  affect  the  quality  of  hospital  care  in  the  United 
States  have  occurred  over  the  past  ten  years  in  addition  to  PPS.  The  number  of  patients 
treated  in  outpatient  settings  has  risen,  and  the  average  burden  of  illness  of  such  patients 
has  also  increased.  In  a  reaction  against  extending  life  in  a  futile  state,  the  use  of  DNR 
orders  has  grown  (Scheidermayer,  1988;  and  Bedell  and  Delbanco,  1984).  PROs  were  estab- 
lished and  have  been  extending  their  reach  and  insisting  on  better  accountability.  New  tech- 
nologies and  medical  knowledge  have  become  available,  and  as  time  passes,  older  physicians 
are  continually  being  replaced  by  younger  physicians  who  have  more  thorough  training  in 
the  use  of  the  new  methods. 

We  report  in  this  section  on  three  types  of  analyses  we  have  conducted  to  supplement 
our  previous  descriptive  before-after  comparisons  with  judgments  as  to  which  of  the  observed 
changes  were  caused  by  PPS.  First,  we  estimate  the  impact  on  mortality  following  hospital- 
ization of  each  of  the  changes  from  pre-  to  post-PPS  described  above.  Second,  for  factors 
where  changes  appear  to  be  substantial,  we  examine  patterns  of  change  among  patient  sub- 
sets, to  see  whether  the  changes  are  consistent  or  concentrated  in  particular  types  of  patients 
and  hospitals.  Third,  we  extend  our  before-after  comparisons  to  a  time  series  analysis  with 
multiple  time  points  pre-  and  post-PPS,  examining  trends  within  and  across  both  periods  to 
see  if  post-PPS  values  of  important  quality  of  care  measures  are  consistent  with  pre-PPS 
trends. 
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METHODS 

Estimating  the  Impact  of  Pre-  and  Post-PPS  Changes  on  Mortality 

In  the  first  of  our  three  methods  for  establishing  which  changes  were  likely  to  have  been 
caused  by  PPS,  we  identified  22  variables,  each  of  which  by  itself  is  associated  with  changes 
in  mortality  30  and  180  days  postadmission.  Table  8.1  summarizes  and  defines  these  vari- 
ables. The  first  five  variables  in  Table  8.1  measure  demographics:  age,  gender,  Medicaid 
status,  race,  and  admission  from  a  nursing  home.  The  next  six  variables  directly  or 
indirectly  measure  patient  sickness  at  admission:  preadmission  functional  status;  two  scales 
we  developed  to  measure  acute  (generally  correctable)  and  chronic  (generally  not  correctable) 
illness  at  admission;  two  additional  scales  we  created  (Sec.  VII)  for  predicting  death  within 
30  and  180  days  of  admission  from  symptoms,  signs,  laboratory  values,  and  diagnoses 
present  at  admission;  and  the  presence  of  a  DNR  order  on  days  1  or  2  of  the  hospital  stay. 
The  next  seven  variables  measure  the  processes  of  care  given  to  the  patient  during  the  hospi- 
tal stay:  five  scales  we  developed  (Sec.  IV)  that  measure  physician  cognitive  diagnostic 
processes,  nurse  cognitive  diagnostic  processes,  technical  diagnostic  processes,  technical 
therapeutic  processes,  and  use  of  telemetry  or  ICU  monitoring;  an  overall  scale  summarizing 
these  five  component  process  scales;  and  the  presence  of  a  DNR  order  written  on  the  third  or 
later  day  of  the  hospitalization  (considered  as  a  process  measure).  The  next  three  variables 
are  scales  we  created  (Sec.  VI)  measuring  instability  at  discharge  (i.e.,  generally  correctable 
symptoms  or  signs  that  were  not  corrected),  sickness  at  discharge  (i.e.,  symptoms  or  signs  of 
illness  that  were  not  necessarily  correctable),  and  very  abnormal  last  laboratory  values 
obtained  during  the  hospital  stay.  The  last  variable  in  Table  8.1  measures  whether  patients 
were  discharged  to  home  or  to  an  institution  (e.g.,  a  skilled  nursing  facility)  (Sec.  VII). 

We  conducted  two  types  of  mortality  analyses  with  these  variables.  In  the  first 
analysis,  we  measured  the  size  of  the  changes  from  pre-  to  post-PPS  in  each  variable,  and 
used  linear  regression  to  estimate  the  change  in  both  30-day  and  180-day  mortality  associ- 
ated with  a  change  of  that  magnitude.  In  the  second  analysis,  we  postulated  a  sequence  of 
effects  corresponding  to  the  temporal  order  in  which  the  variables  potentially  affect  the 
patient  before  and  during  the  hospital  stay.  In  this  temporal  order,  the  five  demographic 
variables  came  first  (introduced  simultaneously),  followed  by:  our  180-day  sickness-at- 
admission  scale  (the  single  admission  sickness  measure  most  highly  correlated  with  long- 
term  death);  initial  DNR  status;  our  summary  inhospital  process  measure;  subsequent  DNR 
status;  and  four  discharge  status  variables:  instability  at  discharge,  abnormal  last  laboratory 
values,  sickness  at  discharge,  and  discharge  destination.  We  computed  the  expected  change 
in  180-day  mortality  associated  with  the  change  in  each  variable,  adjusting  for  all  earlier 
variables  in  the  temporal  sequence.  We  provide  details  on  the  calculations  leading  to  these 
estimates  below.  Note  that  these  were  comparative  analyses,  designed  to  tell  us  which  fac- 
tors were  potentially  the  most  influential  on  death,  and  that  as  such  they  are  sufficient  to 
demonstrate  association  but  not  necessarily  causation.  Note  also  that  sequences  of  variables 
different  from  the  ordering  we  chose  would  produce  different  effect  estimates  for  the  vari- 
ables in  our  analysis  (except  for  demographics,  which  were  entered  as  a  block  of  variables  at 
the  beginning  of  the  sequence). 

Methods  for  Estimating  the  Effects  on  Mortality  of  Changes  in  Variables  from 
Pre-  to  Post-PPS.  In  the  first  part  of  our  mortality  audit,  we  took  each  variable  listed  in 
Table  8.1  through  three  steps.  First  we  noted  how  the  variable  changed  from  pre-  to  post- 
PPS.  Second,  we  used  simple  linear  regression  of  mortality  on  each  variable  to  assess  the 
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Table  8.1 

VARIABLES  USED  TO  STUDY  EFFECTS  OF  PPS  ON  MORTALITY 
FOLLOWING  HOSPITALIZATION 


Age:  Age  in  years 

Gender:  Female  (scored  1),  male  (scored  0) 

Race:  Black  (scored  1),  other  (scored  0) 

Medicaid:  Had  Medicaid  (scored  1),  all  others  (scored  0) 

Preadmission  nursing  home  residence:  The  patient  resided  in  a  nursing  home  immediately  before  the 
study  hospitalization  (scored  1),  all  others  (scored  0).  "Nursing  home"  is  denned  as  a  skilled  nursing  facil- 
ity, intermediate  care  facility,  extended  care  facility,  or  nursing  home  unspecified. 

30-day  mortality  prediction:  A  disease-specific  weighted  sum  of  variables  selected  by  logistic  regres- 
sion of  30-day  postadmission  death  on  clinical  conditions  present  at  the  time  of  the  study  hospitalization. 
See  Sec.  Ill  for  details. 

180-day  mortality  prediction:  A  disease-specific  weighted  sum  of  variables  selected  by  logistic  regres- 
sion of  180-day  postadmission  death  on  clinical  conditions  present  before  admission  for  the  study  hospital- 
ization. See  Sec.  Ill  for  details. 

Initial  DNR:  A  DNR  order  was  written  by  the  physician  on  days  1  or  2  of  the  study  hospitalization 
(scored  1),  all  others  (scored  0). 

Process:  Five  standardized  process  scales  summarizing  disease-specific  process  measures  including  phy- 
sician cognitive  diagnostic  processes,  nurse  cognitive  diagnostic  processes,  technical  diagnostic  processes, 
technical  therapeutic  processes,  and  ICU/telemetry  monitoring,  plus  an  overall  summary  scale.  See  Sec. 
rV  for  details. 

Subsequent  DNR:  A  DNR  order  was  written  by  the  physician  on  day  3  or  later  of  the  study  hospitaliza- 
tion (scored  1),  all  others  (scored  0). 

Instability  at  discharge:  Scored  1  if  there  was  at  least  one  clinical  problem  noted  at  discharge  (a)  that 
should  have  been  corrected  before  discharge  and  (b)  that  was  likely  to  be  associated  with  worse  patient 
outcomes,  all  others  (scored  0).  See  Sec.  VI  for  details. 

Abnormal  last  labs:  Scored  1  if  there  was  at  least  one  dangerously  abnormal  laboratory  value  the  last 
time  laboratory  values  were  recorded  during  the  study  hospitalization,  all  others  (scored  0).  See  Sec.  VI 
for  details. 

Sickness  at  discharge:  Scored  1  if  there  was  at  least  one  measure  of  clinical  sickness  at  the  time  of  hos- 
pital discharge  that  included,  but  was  not  limited  to,  problems  that  should  have  been  corrected  before 
discharge,  all  others  (scored  0).  See  Sec.  VI  for  details. 

Discharge  destination:  The  patient's  discharge  destination  was  scored  1  if  the  patient  was  discharged 
to  an  institution,  denned  to  include  nursing  homes  (skilled,  intermediate,  extended  care  facilities,  or 
unspecified);  retirement  homes  (sheltered  housing,  congregate  housing,  halfway  house,  or  board  and  care 
facilities);  psychiatric  facilities  (mental  hospitals);  chronic  disease  hospitals;  or  rehabilitation  hospitals;  all 
others  (scored  0).  See  Sec.  VII  for  details. 


NOTE:  All  variables  were  abstracted  from  the  medical  record. 


121 


effect  of  the  variable  on  death  rates  30  and  180  days  after  admission.  Third,  we  put  these 
two  steps  together  multiplicatively  to  estimate  the  expected  contribution  of  pre-  and  post- 
PPS  changes  in  each  variable  to  death  rates,  unadjusted  for  any  other  variables.  For  exam- 
ple, for  congestive  heart  failure  we  found  that  the  summary  processs  score  was  0.42  standard 
deviation  units  higher  post-PPS  than  pre-PPS.  An  increase  of  1  standard  deviation  unit  in 
this  measure  was  associated  with  a  4.4  percentage  point  reduction  in  30-day  postadmission 
mortality  for  congestive  heart  failure  patients.  Thus  we  estimate  the  expected  effect  of  the 
pre-  and  post-PPS  change  in  process  on  30-day  mortality  for  congestive  heart  failure 
patients,  if  nothing  else  had  changed,  to  be  a  mortality  reduction  of  0.42  x  4.4  =1.8  percen- 
tage points.  This  analytic  method  has  been  described  in  more  detail  by  Wonnacott  and  Won- 
nacott  (1984). 

In  the  second  part  of  the  mortality  audit,  we  postulated  a  sequence  of  effects 
corresponding  to  the  temporal  order  in  which  factors  potentially  affect  the  patient  arriving  at 
the  hospital.  We  began  with  a  dummy  variable  for  pre-  and  post-PPS  and  then  added  to  the 
model  demographics,  sickness  at  admission,  initial  DNR,  process,  subsequent  DNR,  and 
discharge  variable  (instability  at  discharge,  sickness  at  discharge,  very  abnormal  last  labora- 
tory values,  and  discharge  to  a  nursing  home).  It  is  important  to  note  that,  with  this 
approach,  attributions  of  the  effect  of  pre-  and  post-PPS  changes  on  mortality  will  depend 
upon  the  sequencing  of  factors.  Changing  the  order  of  demographics  and  sickness  at  admis- 
sion, for  example,  would  change  the  perception  of  which  of  these  two  is  responsible  for 
changes  in  death  resulting  from  sickness  at  admission  but  would  not  affect  other  conclusions. 

To  evaluate  the  effects  of  demographics,  sickness  at  admission,  and  inhospital  processes 
of  care,  we  used  linear  regression  models.  Standard  errors  for  effect  estimates  were  deter- 
mined by  propagation  of  error  (Yasukawa,  1930).  For  example,  the  coefficient  of  process  for 
congestive  heart  failure  in  our  linear  regression  of  30-day  mortality  on  demographics,  sick- 
ness at  admission,  initial  DNR,  and  process  was  -0.024  with  a  standard  error  of  0.008,  and 
the  pre-  post-PPS  improvement  in  process  (adjusting  for  the  variables  before  process  in  the 
temporal  sequence)  was  0.43  with  a  standard  error  of  0.036.  Thus  the  estimated  adjusted 
effect  of  the  pre-  post-PPS  change  in  process  on  30-day  mortality  was  (-0.024  x  0.43)  =  -1.0 
percentage  points,  and  the  standard  error  of  this  estimate  was  [(0.036  x  0.024)  x  2  +  (0.008  x 
0.43)  x  2]  =  0.37  percent. 

To  assess  the  impact  of  the  discharge  status  variables  on  postdischarge  mortality,  we 
used  Cox  proportional  hazards  time-varying  regression  (Kalbfleisch  and  Prentice,  1980). 
This  model  predicts  patient  mortality  on  each  day  after  admission  as  a  function  of  all  of  the 
admission  variables.  A  time-varying  dummy  variable  is  used  to  account  for  the  different 
mortality  rates  of  patients  inside  and  outside  the  hospital.  Discharge  variables  are  intro- 
duced as  of  the  date  of  discharge.  This  method  accounts  for  the  decrease  in  length  of  stay 
associated  with  the  introduction  of  PPS  and  the  decrease  in  mortality  as  a  function  of  time 
after  admission,  up  to  at  least  60  days.  The  Cox  model  gives  the  relative  probability  of  death 
for  each  variable.  To  get  its  actual  impact,  we  multiplied  this  effect  by  the  average  probabil- 
ity of  death  between  discharge  and  180  days. 

Patient  and  Hospital  Subset  Analysis  Methods.  We  used  regression  to  see 
whether  the  changes  from  pre-  to  post-PPS  in  our  summary  process  measure,  initial  DNR 
use,  and  instability  at  discharge  were  significantly  different  for  subgroups  of  patients  denned 
by  hospital-  and  patient-level  grouping  variables.  For  example,  to  assess  the  significance  of 
the  effect  on  process  of  the  categorical  city  size/teaching  variable  described  in  Methods, 
above,  with  levels  of  rural  nonteaching,  urban  nonteaching,  and  urban  teaching,  we  linearly 
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regressed  process  on  a  dummy  variable  for  pre-  post-PPS,  our  30-day  sickness-at-admission 
scale,  a  set  of  two  dummy  variables  defining  city  size/teaching  status,  and  a  set  of  two 
dummy  variables  representing  the  interaction  between  pre-  post-PPS  and  city  size/teaching. 
The  adjustment  for  sickness  at  admission  had  little  effect,  so  we  report  unadjusted  means  in 
Table  8.3,  below.  Logistic  regression  was  used  instead  of  linear  regression  for  initial  DNR 
status  and  the  presence  or  absence  of  one  or  more  instabilities  at  discharge. 

Trend  Analysis  Methods.  We  used  simple  time  series  methods  to  investigate 
whether  post-PPS  values  of  30-day  mortality  (adjusted  for  sickness  at  admission),  overall 
inhospital  process,  instability  at  discharge,  and  initial  use  of  DNR  orders  were  reasonably 
consistent  with  pre-PPS  trends.  To  do  this,  we  calculated  quarterly  means,  aggregating 
across  the  five  diseases,  of  each  of  these  measures  and  made  time  series  plots  of  the  mean 
values,  with  traces  two  standard  errors  either  way  from  the  means  superimposed  to  illus- 
trate the  sampling  variability.  We  then  explored  a  variety  of  simple  models  extrapolating  the 
pre-PPS  trends  to  the  post-PPS  period  and  fitting  the  observed  post-PPS  trends,  including 
constant,  linear,  and  quadratic  time  effects.  Constant  and  linear  trends  were  sufficient  to 
describe  the  behaviors  observed  in  Figs.  8.1-8.4,  below.  For  at  least  two  reasons — our  data 
go  back  only  to  1981,  and  there  is  substantial  forecasting  uncertainty  internal  to  a  single 
extrapolation  model,  as  well  as  uncertainty  about  the  form  that  such  a  model  should  take — it 
should  not  be  concluded  that  these  methods  are  sufficient  to  definitively  estimate  what  qual- 
ity of  care  would  have  been  like  in  the  post-PPS  period  in  the  absence  of  PPS.  Nor  are  they 
intended  to:  They  serve  only  to  help  confirm  or  deny  PPS's  role  in  causing  the  observed 
differences. 

In  the  analysis  of  the  sources  of  change,  we  also  tested  interactions  between  the  vari- 
ables in  this  analysis  and  those  in  pre-  and  post-PPS  periods.  If  present,  these  interactions 
would  signify  changes  in  the  relationship  between  these  factors  and  death.  An  F-test  was 
used. 

Patient  and  Hospital  Subset  Analyses 

With  our  second  method,  we  examined  patient  and  hospital  subsets  to  see  whether  pre- 
and  post-PPS  differences  occurred  globally  across  all  subsets  or  were  restricted  to  certain 
patient  groups.  Underlying  this  analysis  was  the  idea  that  a  large  systemic  change,  such  as 
Medicare's  movement  from  retrospective  to  prospective  payment,  is  likely  to  have  had  per- 
vasive effects  that  appear  consistently  across  a  wide  variety  of  patient  and  hospital  types. 
Less  global  changes,  such  as  diffusion  of  current  standards  of  medical  practice  from  urban  to 
rural  hospitals,  that  took  place  at  about  the  same  time  as  the  introduction  of  PPS  might  have 
had  isolated  effects  that  were  concentrated  in  certain  classes  of  patients  and  hospitals. 

In  this  analysis,  we  examined  how  process,  initial  DNR  use,  and  discharge  status 
changed  across  patient  subsets  defined  by  a  variety  of  patient-  and  hospital-level  factors. 
The  patient  subsets  we  studied  differed  by  levels  of  sickness  and  functioning,  and  the  hospi- 
tal subsets  by  city  size,  state,  percentage  of  Medicaid  patients,  and  teaching  status.  We  com- 
puted separately  for  each  of  our  five  diseases  the  changes  from  pre-  to  post-PPS  in  initial  use 
of  DNR  orders,  our  overall  process  of  care  scale,  and  instability  at  discharge  for  each  subset 
of  the  city  size/teaching  variable,  separately  for  each  of  the  five  diseases,  and  used  linear 
regression  to  assess  the  significance  of  these  differences  (see  above).  Next,  we  developed  a 
hospital-level  variable  that  combined  city  size  and  teaching  as  follows:  (1)  rural  nonteaching, 
(2)  urban  nonteaching,  and  (3)  urban  teaching,  because  these  were  the  most  interesting  dis- 
tinctions. (The  category,  rural  teaching,  had  virtually  no  hospitals.) 
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We  used  linear  regression  to  assess  the  significance  of  differences  in  process,  initial 
DNR  use,  and  discharge  status  for  patients  in  these  three  subsets. 

Trend  Analyses 

Finally,  we  made  graphical  comparisons  between  (a)  the  actual  values  of  important 
quality  of  care  variables  in  1985—1986  and  (b)  simple  estimates  of  what  those  values  might 
have  been  if  PPS  had  not  been  implemented.  To  do  this  we  used  our  patient-level  data  to  cal- 
culate quarterly  summaries  of  four  variables:  180-day  mortality,  adjusted  for  sickness  at 
admission;  overall  inhospital  process;  instability  at  discharge;  and  initial  use  of  DNR  orders. 
We  plotted  these  quarterly  values  on  the  same  graphs  with  predictions  based  on  simple 
extrapolations  from  the  pre-PPS  period  (see  above).  With  only  eight  quarters  of  data  in  the 
pre-PPS  period  and  with  PPS  introduced  nonexperimentally,  our  ability  to  definitively  say 
what  quality  of  care  would  have  been  like  in  the  post-PPS  period  in  the  absence  of  PPS  is  lim- 
ited (for  example,  the  confidence  bands  based  on  linear  extrapolation  models  are  quite  wide). 
However,  if  changes  between  1981-1982  and  1985-1986  are  consistent  with  trends  within 
those  two  time  periods,  it  would  appear  that  the  changes  are  a  continuation  of  trends  in  place 
historically  and  not  a  specific  result  of  PPS. 

RESULTS 

The  Analysis  of  the  Sources  of  Change 

Table  8.2  shows  how  each  variable  in  Table  8.1  changed  from  the  pre-PPS  to  the  post- 
PPS  period,  and  how  these  differences  are  associated  with  changes  in  mortality  rates  30  and 
180  days  after  admission.  Each  row  in  Table  8.2  has  four  values:  the  pre-PPS  mean  of  the 
variable  defining  that  row,  the  mean  change  in  that  variable  from  pre-  to  post-PPS,  and  the 
expected  changes  in  mortality  30  and  180  days  from  admission  associated  with  that 
variable's  pre-  and  post-PPS  difference.  This  table  displays  the  independent  effect  on  mortal- 
ity of  the  pre-  and  post-PPS  difference  in  each  measure,  unadjusted  for  any  other  variables. 
For  example,  Table  8.2  shows  that  the  average  age  of  congestive  heart  failure  patients 
increased,  but  not  significantly,  from  78.3  years  pre-PPS  to  78.3  +  0.3  =  78.6  years  post-PPS. 
Given  the  relationship  between  age  and  mortality,  this  increase  in  age  translates,  without 
adjusting  for  any  other  variables,  into  an  0.1  percentage  point  expected  increase  in  both  30- 
day  and  180-day  postadmission  mortality. 

Among  the  demographic  variables,  age  increased  post-PPS  for  the  four  medical  diseases 
(p  <  0.05  for  acute  myocardial  infarction  patients)  but  stayed  about  the  same  for  hip  fracture 
patients.  Preadmission  residence  in  a  nursing  home  also  increased  for  patients  with  the  four 
medical  diseases  (p  <  0.05  for  two  of  the  four  medical  conditions)  but  decreased  significantly 
for  patients  with  hip  fracture  (p  <  0.05). 

The  six  sickness  variables  show  that  patients  were  noticeably  sicker  at  admission  in 
1985-1986  than  in  1981-1982  (especially  pneumonia  patients),  a  difference  that  is  associated 
with  higher  mortality  rates  post-PPS.  Three  of  these  measures — acute  sickness,  chronic  sick- 
ness, and  functional  status — are  uncorrected  for  changes  in  recording  and  laboratory  order- 
ing between  the  pre-  and  post-PPS  periods.  When  we  adjust  for  these  changes  and  use 
recording-insensitive  criteria  (such  as  our  30-day  and  180-day  sickness  scales),  we  still  find 
that  patients  are  sicker  post-PPS,  but  the  differences  are  less  pronounced  (Sec.  III).  We  also 


Table  8.2 


CHANGES  IN  22  VARIABLES  PRE-  AND  POST-PPS  AND  EXPECTED  DIFFERENCES  IN  MORTALITY 
PER  100  PATIENTS  ASSOCIATED  WITH  THESE  CHANGES 


Pre-PPS 

Expected  Change  from  Pre-  and 

Values 

Differences 

Post-PPS  Changes  in  Variables* 

(in  Original 

(Post-PPS 

Measures 

Units) 

-  Pre-PPS)     30-Day  Mortality 

180-Day  Mortalityb 

Congestive  Heart  Failure 

Age  (years) 

78.3 

+0.3 

0.1 

0.1 

Gender  (%  female) 

57.1 

-2.2 

0.0 

0.0 

Medicaid  patient  (%) 

1  H  A 

17. y 

A  £. 

-U.o 

A  A 
U.U 

A  A 
U.U 

Black  (%\ 

14.3 

-1.7 

0  2 

0.2 

Preadmission  nursinc*  home  f9M 

X  1  vUUXlllOOiull   UU1  Jlilg  11  \J  lilt*   1   /I/  1 

8.9 

+2.0 

0.2 

0.3 

Function 

1.6 

+0.3* 

0.2 

0.5 

Acute  sickness 

2.2 

+0.3* 

0.6 

0.8 

Chronic  sickness 

4.7 

+0.4** 

0.3 

0.9 

30-day  sickness  scale 

32.3 

+0.2 

0.2 

0.5 

180-day  sickness  scale 

38.3 

+0.3 

0.3 

0.6 

Initial  DNR  (%) 

3.8 

+1.7* 

0.4 

0.6 

ICU/telemetry  process 

-0.13 

+0.2* 

-0.3 

-1.2 

Physician  cognitive  process 

-0.16 

+0.3** 

-0.9 

-1.2 

Nurse  cognitive  process 

-0.20 

+0.4** 

-1.4 

-1.5 

Technical  diagnostic  process 

-0.06 

+0.3** 

-0.2 

-0.2 

Technical  therapeutic  process 

-0.13 

+0.1* 

-0.5 

-0.3 

Overall  process 

-0.22 

+0.4** 

-1.8 

-2.2 

Subsequent  DNR  (%) 

1.4 

+0.5 

0.0 

0.2 

Abnormal  last  lab  (%) 

22.9 

-1.3 

-0.0 

-0.1 

Instability  at  discharge  (%) 

7.3 

+1.7* 

0.1 

0.2 

Sickness  at  discharge  (%) 

21.8 

+5.4** 

0.2 

0.6 

Discharge  to  a  nursing  home  (%) 

6.9 

+1.6* 

0.1 

0.3 

Acute  Myocardial  Infarction 

Age  (years) 

75.4 

+0.7* 

0.6 

0.9 

Gender  (%  female) 

47.0 

+2.9 

0.0 

0.1 

Medicaid  patient  (%) 

13.6 

+0.1 

0.0 

0.0 

Black  (%) 

8.2 

+0.6 

-0.0 

-0.0 

Preadmission  nursing  home  (%) 

4.8 

+2.8* 

0.3 

0.5 

Function 

0.4 

+1.2** 

0.5 

0.9 

Acute  sickness 

6.3 

+0.8** 

1.9 

2.3 

Chronic  sickness 

0.91 

+0.3** 

0.6 

1.2 

30-day  sickness  scale 

25.1 

+1.0 

1.4 

1.7 

180-day  sickness  scale 

39.2 

+1.4 

1.5 

2.0 

Initial  DNR  (%) 

2.4 

+0.3** 

1.3 

2.0 

ICU/telemetry  process 

-0.03 

+0.05 

-0.1 

-0.1 

Physician  cognitive  process 

-0.15 

+0.23  ** 

-1.7 

-1.5 

Nurse  cognitive  process 

-0.12 

+0.20** 

-1.4 

-1.5 

Technical  diagnostic  process 

-0.09 

+0.17  ** 

0.2 

0.4 

Technical  therapeutic  process 

-0.12 

+0.20** 

-1.2 

-1.0 

Overall  process 

-0.14 

+0.26  ** 

-1.9 

-1.7 

Subsequent  DNR  (%) 

0.6 

+1.4** 

0.0 

0.3 

Abnormal  last  lab  (%) 

5.6 

0.0 

-0.0 

-0.0 

Instability  at  discharge  (%) 

4.2 

+1.3 

-0.0 

0.0 

Sickness  at  discharge  (%) 

13.3 

+3.4* 

0.1 

0.3 

Discharge  to  a  nursing  home  (%) 

3.7 

+1.1 

0.0 

0.2 

Pneumonia 

Age  (years) 

78.4 

+0.4 

0.3 

0.5 

Gender  (%  female) 

49.2 

+0.3 

-0.0 

-0.0 

Medicaid  (%) 

21.4 

-0.1 

-0.0 

-0.0 

Black  (%) 

8.0 

+1.5 

0.0 

0.1 

Preadmission  nursing  home  (%) 

24.5 

+4.3 

0.7 

1.1 

Table  8.2 — continued 


Pre-PPS 

Expected  Change  from  Pre-  and 

Values 

Differences 

Post-PPS  Changes  in  Variables3 

(in  Original 

(Post-PPS 

u 

Measures 

Units) 

-  Pre-PPS) 

30-Day  Mortality 

180-Day  Mortality 

Function 

1.7 

+0.4** 

1.5 

2.6 

Acute  sickness 

8.2 

+0.8** 

1.9 

2.5 

Chronic  sickness 

4.2 

+1.0** 

1.7 

3.3 

30-day  sickness  scale 

25.8 

+2.6** 

3.4 

4.7 

180-day  sickness  scale 

31.7 

+3.4** 

3.6 

5.4 

Initial  DNR  (%) 

7.1 

+3.9** 

1.3 

2.0 

ICU/telemetry  process 

-0.12 

+0.12* 

0.6 

0.5 

Physician  cognitive  process 

-0.13 

+0.24** 

-0.5 

-0.5 

Nurse  cognitive  process 

-0.20 

+0.38** 

-2.0 

-3.1 

Technical  diagnostic  process 

-0.07 

+0.14** 

-0.3 

-0.4 

Technical  therapeutic  process 

-0.11 

+0.22** 

-0.4 

-0.6 

Overall  process 

-0.22 

+0.42** 

-1.2 

-2.1 

Subsequent  DNR  (%) 

1.9 

+1.5** 

0.1 

0.3 

Abnormal  last  lab  (%) 

13.7 

+2.6* 

0.1 

0.2 

Instability  at  discharge  (%) 

6.7 

+1.7* 

0.0 

0.2 

Sickness  at  discharge  (%) 

24.6 

+12.1** 

0.4 

0.9 

Discharge  to  a  nursing  home  (%) 

11.6 

+4.2** 

0.3 

0.9 

Cerebrovascular  Accident 

Age  (years) 

78.3 

+0.1 

0.1 

0.2 

Gender  (%  female) 

54.6 

+2.0 

0.0 

0.1 

Medicaid  patient  (%) 

16.5 

-2.0 

-0.1 

-0.1 

Black  (%) 

11.7 

+0.1 

-0.0 

0.0 

Preadmission  nursing  home  (%) 

14.1 

+0.4 

0.1 

0.1 

Function 

1.2 

+0.0 

0.2 

0.3 

Acute  sickness 

5.3 

+0.2 

1.1 

1.4 

Chronic  sickness 

3.6 

+0.4** 

0.2 

0.6 

30-day  sickness  scale 

20.2 

-0.5 

-0.8 

-0.9 

180-day  sickness  scale 

29.7 

-0.6 

-0.9 

-1.2 

Initial  DNR  (%) 

7.1 

+3.9** 

1.3 

2.0 

ICU/telemetry  process 

-0.07 

+0.08 

0.2 

0.2 

Physician  cognitive  process 

-0.20 

+0.37** 

-0.7 

-0.6 

Nurse  cognitive  process 

-0.25 

+0.47** 

-0.9 

-1.1 

Technical  diagnostic  process 

-0.13 

+0.26** 

-0.6 

-0.5 

Overall  process 

-0.25 

+0.49** 

-1.1 

-1.2 

Subsequent  DNR  (%) 

4.2 

+0.7 

0.0 

0.1 

Abnormal  last  lab  (%) 

11.5 

-0.5** 

-0.0 

-0.1 

Instability  at  discharge  (%) 

10.0 

+2.0* 

0.1 

0.2 

Sickness  at  discharge  (%) 

12.2 

+2.7* 

0.1 

0.5 

Discharge  to  a  nursing  home  (%) 

21.4 

+2.6* 

0.1 

0.6 

Hip  Fracture 

Age  (years) 

81.5 

-0.1 

-0.0 

-0.1 

Gender  (%  female) 

79.0 

-2.7 

0.1 

0.3 

Medicaid  patient  (%) 

17.7 

-0.6 

-0.0 

-0.0 

Black  (%) 

5.6 

0.0 

0.0 

0.0 

Preadmission  nursing  home  (%) 

28.4 

-3.8* 

-0.1 

-0.4 

Function 

2.2 

0.0 

-0.0 

-0.1 

Acute  sickness 

1.0 

0.0 

0.0 

0.0 

Chronic  sickness 

3.0 

+0.4** 

0.4 

1.0 

30-day  sickness  scale 

7.2 

+0.4** 

0.2 

0.6 

180-day  sickness  scale 

14.6 

+0.7 

0.3 

0.9 

Initial  DNR  (%) 

0.4 

+0.8* 

0.1 

0.2 
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Table  8.2 — continued 

prp  ppc 

rJC-rrj 

Expected  Change  from  Pre-  and 

v  aiucs 

Differences 

Post-PPS  Changes  in  Variables1 

(in  Original 

(Post-PPS 

Measures 

Units) 

-  Pre-PPS)     30-Day  Mortality" 

180-Day  Mortalityb 

T^'T  T  /t  o  1  e» ry\  £*tr\T  nm/^pce 

iv— u/icieinciry  pruccsa 

_n  no 

0.00 

0.0 

U.U 

I  HVoJClali  CUtlliUvt  Jjiuvk-oo 

-0.09 

+0.17** 

0.1 

0  1 

Nnr^p  cfioriitive  nrocess 

-0.16 

+0.31** 

-0.1 

-0.4 

Technical  diagnostic  process 

-0.24 

+0.29** 

-0.1 

-0.2 

Technical  therapeutic  process 

-0.13 

+0.23** 

0.1 

-0.2 

Overall  process 

-0.22 

+0.42** 

0.2 

-0.1 

Subsequent  DNR  (%) 

2.1 

-0.1 

-0.0 

-0.0 

Abnormal  last  lab  (%) 

14.6 

+0.4 

0.0 

0.0 

Instability  at  discharge  (%) 

9.5 

+2.0* 

0.0 

0.1 

Sickness  at  discharge  (%) 

14.8 

+1.8 

0.0 

0.2 

Discharge  to  a  nursing  home  (%) 

29.6 

+1.9 

0.0 

0.3 

NOTE:  *p<0.05,  **p<0.01. 

aExpected  change  in  mortality  from  post-PPS  changes  in  admission  variables  is  computed  by  multiplying 
the  change  in  the  measure  from  pre-PPS  to  post-PPS  by  the  regression  coefficient  of  the  measure  for  predict- 
ing death  (see  above).  These  effects  are  unadjusted  for  changes  in  any  other  measures. 

Expected  change  in  mortality  from  post-PPS  changes  in  discharge  variables  (subsequent  DNR,  instability, 
abnormal  last  lab,  sickness  at  discharge,  discharge  to  a  nursing  home)  is  computed  by  multiplying  the  change 
in  the  measure  from  pre-PPS  to  post-PPS  (excluding  inhospital  deaths  and  acute  care  transfers)  by  the  hazard 
ratio  from  a  proportional  hazards  model  by  the  probability  of  death  between  discharge  and  30  or  180  days 
(see  above). 

A  minus  sign  means  that  predicted  mortality  was  lower  post-PPS  as  a  result  of  the  change  in  the  variable 
defining  a  given  row;  a  plus  sign  means  that  predicted  mortality  was  higher  post-PPS. 


found  significant  increases  in  the  initial  use  of  DNR  orders  in  all  five  study  diseases  (from  7.1 
percent  to  11.0  percent  in  pneumonia,  for  example).  Unadjusted  for  other  variables,  this 
increase  in  the  frequency  with  which  DNR  orders  are  written  is  associated  with  a  substantial 
increase  in  mortality. 

The  five  process  scales  and  the  overall  summary  process  measures  show  dramatic  and 
significant  improvements  during  the  post-PPS  period  for  all  diseases  (p  <  0.01).  These 
improvements  are  associated  with  a  large  decline  in  mortality.  The  frequency  of  DNR  orders 
written  on  day  3  or  later  increased  post-PPS  (significantly  for  acute  myocardial  infarction 
and  pneumonia,  p  <  0.01),  but  such  orders  remain  rare.  Sickness  at  discharge  increased 
post-PPS,  significantly  for  all  diseases  except  hip  fracture;  the  increase  was  particularly 
large  for  pneumonia  patients  (pre-PPS  24.6  percent  of  pneumonia  patients  had  at  least  one 
sickness  indicator  present  at  discharge,  compared  with  36.7  percent  post-PPS).  Instability  at 
discharge  also  increased  significantly  for  all  diseases  except  myocardial  infarction.  These 
changes  are  associated  with  higher  mortality  post-PPS. 

Table  8.3  presents  data  similar  to  those  given  in  Table  8.2  for  the  effects  of  changes  in 
summary  variables  from  Table  8.2  on  180-day  postadmission  mortality.  However,  in  Table 
8.3  the  estimates  were  adjusted  for  prior  variables  in  the  temporal  sequence,  whereas  in 
Table  8.2  the  estimates  were  unadjusted.  For  example,  in  Table  8.3,  the  effect  of  changes  in 
process  on  mortality  takes  account  of  the  effects  of  changes  in  demographics,  sickness  at 
admission,  and  initial  use  of  DNR  orders  on  mortality.  The  effects  on  30-day  mortality  are 
very  similar,  so  they  are  not  shown. 


Table  8.3 


SUMMARY  OF  PREDICTED  CHANGES  IN  180-DAY  POSTADMISSION 
MORTALITY  PER  100  PATIENTS  FROM  CHANGES  PRE-  TO 
POST-PPS  IN  14  VARIABLES 


CHF 

AMI 

PNE 

CVA 

HIP 

Patient  Demographics8 

0.7** 

1.2** 

1.4** 

0.3 

-0.2 

Ape 

0.1 

0.8** 

0.3 

0.2 

-0.1 

Gender  (female) 

0.2 

0.0 

0.0 

0.0 

0.3 

Race  (black) 

0.1 

-0.0 

0.1 

0.0 

0.0 

Medicaid 

0.0 

0.0 

0.0 

0.0 

0.0 

Preadmission  residence 

0.3 

0.4* 

1.0** 

0.1 

-0.4* 

Sickness  at  admission*3 

-0.1 

0.5 

3.0** 

-1.1 

1.1* 

Initial  DNR 

0.3 

0.7** 

0.5* 

0.5* 

0.2 

Process 

-1.0** 

-0.7** 

-1.3** 

-1.0* 

-0.5 

Subsequent  DNR 

0.2 

0.1 

0.2 

0.0 

-0.0 

Discharge  problems0 

0.6** 

0.2 

0.9** 

0.6** 

0.4 

Instability  at  discharge 

0.2 

0.0 

0.2 

0.5* 

0.1* 

Abnormal  last  lab 

-0.2 

-0.0 

0.2* 

-0.1 

0.0 

Sickness  at  discharge 

0.4* 

0.2 

0.3 

0.1 

-0.0 

Discharge  destination 

0.2 

-0.0 

0.2* 

0.1 

0.3* 

Total  change 

accounted  for** 

0.7 

2.0 

4.7 

-0.7 

1.0 

Actual  differences  in  observed 

death  rates  (per  100  patients)6 

-1.1 

3.4 

5.8 

-1.2 

-1.9 

Remaining  unexplained  differences 

-1.9 

1.4 

1.1 

-0.6 

-3.0 

NOTES:  *  p  <  0.05,  **  p  <  0.01.  Except  for  demographics,  the  effect  of 
each  variable  is  the  added  effect,  controlling  for  variables  higher  on  the  list. 
The  mortality  effects  displayed  here  are  an  average  of  the  pre-PPS  period,  in 
which  mortality  effects  were  large,  and  the  post-PPS  period,  where  they  were 
somewhat  smaller. 


Predicted  changes  for  patient  demographics  are  a  sum  of  the  predicted 
changes  for  age,  gender,  race,  Medicaid,  and  preadmission  residence. 

Sickness  at  admission  is  evaluated  using  two  variables:  scale  designed 
to  predict  death  at  30  days  and  a  scale  designed  to  predict  death  at  180  days 
(Table  8.1). 

cPredicted  changes  for  discharge  problems  are  a  sum  of  the  predicted 
changes  for  instability  at  discharge,  abnormal  last  laboratory  value,  sickness 
at  discharge,  and  discharge  destination. 

^his  row  describes  the  total  change  post-PPS  -  pre-PPS  in  180-day  post- 
admission  mortality  rates  that  are  accounted  for  by  the  post-PPS  -  pre-PPS 
changes  in  the  13  variables  listed  above.  For  example,  for  CHF,  the  sum  of 
the  percentages  listed  above  is  0.8  percent,  meaning  that  mortality  associ- 
ated with  the  named  factors  is  estimated  to  increase  by  a  total  of  0.8  percent- 
age points.  The  actual  observed  change  in  death  rates  is  -1.1  percent  and  the 
difference  (-1.1-0.8)  is  due  to  other  unnamed  causes. 

eActual  differences  in  observed  death  rates  when  comparing  180-day 
unadjusted  postadmission  mortality  rates  post-PPS  with  pre-PPS:  CHF  33.0 
percent  pre-PPS,  31.9  percent  post-PPS;  AMI  32.4  percent  pre,  35.9  percent 
post;  PNE  25.6  percent  pre,  31.3  percent  post;  CVA  35.4  percent  pre,  34.2 
percent  post;  HIP  17.2  percent  pre,  15.3  percent  post. 
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The  major  effects  on  180-day  mortality  associated  with  pre-  and  post-PPS  changes  in 
the  variables  in  Table  8.3  are  concentrated  in  four  variables:  sickness  at  admission,  initial 
DNR,  inhospital  process  of  care,  and  discharge  status.  Post-PPS  increases  in  admission  sick- 
ness, adjusting  for  changes  in  demographics,  had  an  especially  pronounced  predicted  effect  in 
pneumonia  (an  expected  rise  of  3.0  percentage  points  in  180-day  mortality)  and  hip  fracture. 
After  adjusting  for  demographics  and  admission  sickness,  the  post-PPS  rise  in  the  initial  use 
of  DNR  orders  was  associated  with  a  modest  increase  (0.2-0.7  percentage  points)  in  expected 
mortality  for  all  five  diseases. 

Improvements  in  inhospital  process  of  care  post-PPS  (after  accounting  for  changes  in 
demographics,  sickness  at  admission,  and  initial  DNR  orders)  were  associated  with  notice- 
able decreases  in  expected  mortality  for  all  five  diseases  (0.5-1.3  percentage  points).  These 
changes  were  significant  for  all  conditions  but  hip  fracture.  There  was  a  statistically 
significant  rise  in  expected  mortality  associated  with  adverse  changes  in  one  or  more 
discharge  variables  in  four  of  the  five  diseases.  Potentially  remediable  changes  in  discharge 
instability  and  abnormal  last  laboratory  values  were  significant  for  three  of  the  five  diseases. 
Taken  together,  discharge  problems  post-PPS  have  increased  enough  to  raise  expected  death 
rates  by  an  amount  that  varies  from  a  total  of  0.2  percentage  points  for  acute  myocardial 
infarction  patients  to  0.9  percentage  points  for  pneumonia  patients. 

Generally,  interactions  between  the  variables  in  Table  8.3  and  time  period  were  statisti- 
cally insignificant.  But  for  instability,  the  mortality  differential  associated  with  unstable 
discharge  in  1985-1986  was  only  about  60  percent  as  large  as  it  was  in  1981-1982.  This 
interaction  effect  was  insignificant  for  individual  diseases  but  significant  (p  <  0.05)  when 
combined  across  all  five  diseases.  Two  hypotheses  come  to  mind.  First,  this  could  be  a  real 
change  in  the  effect  of  disability  over  time.  Second,  this  could  have  reflected  a  change  in  the 
definition  or  use  of  instability  measures.  However,  one  must  consider  the  second  hypothesis 
in  light  of  the  finding  that  recording-sensitive  symptoms  and  recording-insensitive  vital  signs 
both  increased  post-PPS  (Sec.  VI). 

In  Table  8.3,  we  present  the  actual  changes  in  raw  death  rates,  the  total  of  all 
accounted-for  expected  changes,  and  the  difference  between  these  two  figures.  It  is  evident 
that  this  analysis  explains  only  a  portion  of  the  actual  differences.  The  discrepancies  could 
be  due  either  to  sampling  variability  (the  standard  errors  for  the  pre-  and  post-PPS  changes 
in  raw  death  rates  are  about  2  percentage  points  for  each  disease)  or  to  systematic  effects 
unaccounted  for  in  our  analysis  of  the  sources  of  change.  For  example,  our  analysis  does  not 
address  the  contribution  of  new  technology  to  lower  death  rates,  because  we  specifically 
excluded  new  technologies  from  our  process  measures. 

Patient  and  Hospital  Subset  Analyses 

Table  8.4  presents  differences  (post-PPS  minus  pre-PPS)  in  our  summary  overall  inhos- 
pital process  score  (in  standard  deviation  units),  the  percentages  of  patients  with  a  DNR 
order  on  admission,  and  the  percentages  of  patients  with  one  or  more  instabilities  at 
discharge,  for  each  of  the  three  levels  of  the  city  size/teaching  hospital  variable  we  created 
(rural  nonteaching,  urban  nonteaching,  and  urban  teaching).  Across  all  five  diseases,  we 
observed  a  consistent  and  significant  pattern  of  differences  by  hospital  setting  in  post-PPS 
improvements  in  explicit  process  of  care.  Rural  nonteaching  hospitals  showed  the  biggest 
gains  in  inhospital  process  of  care  and  urban  teaching  facilities  the  least. 
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Table  8.4 

DIFFERENCES  (POST-PPS  MINUS  PRE-PPS)  IN  PROCESS,  INITIAL  DNR,  AND  INSTABILITY 
AT  DISCHARGE,  BY  CITY  SIZE  AND  TEACHING  STATUS  OF  HOSPITAL 


Rural  Urban 
Characteristics                     Nonteaching  Nonteaching 

Urban 
Teaching 

Statistical 
Significance 

Congestive  Heart  Failure  (n  = 

2,812) 

Sample  sizes  644 
Process  (standard  deviation  units)  +0.73 
Initial  DNR  (percentage  points)  +3.1 
Instability  at  discharge  (percentage  points)  +4.3 

1,182 
+0.38 
+0.0 
+3.5 

986 
+0.27 
+2.7 
+2.8 

p  <0.01 
NS 
NS 

Acute  Myocardial  Infarction  (n 

=  2,841) 

Sample  sizes  623 
Process  (standard  deviation  units)  +0.40 
Initial  DNR  (percentage  points)  +3.7 
Instability  at  discharge  (percentage  points)  +1.7 

1,238 
+0.28 
+3.0 
+1.1 

980 
+0.17 
+2.4 
+4.7 

p  <0.05 
NS 
NS 

Pneumonia  (n  =  2,727) 

Sample  sizes  600 
Process  (standard  deviation  units)  +0.56 
Initial  DNR  (percentage  points)  +3.5 
Instability  at  discharge  (percentage  points)  +11.6 

1,169 
+0.41 
+4.4 
-0.3 

958 
+0.39 
+3.5 
+2.5 

NS 
NS 
p  <0.10 

Cerebrovascular  Accident  (n  = 

2,810) 

Sample  sizes  601 
Process  (standard  deviation  units)  +0.65 
Initial  DNR  (percentage  points)  +1.4 
Instability  at  discharge  (percentage  points)  +6.3 

1,212 
+0.49 
+1.9 
+3.7 

997 
+0.39 
+4.0 
+2.3 

p  <0.05 
NS 
NS 

Hip  Fracture  (n  =  2,746) 

Sample  sizes  468 
Process  (standard  deviation  units)  +0.58 
Initial  DNR  (percentage  points)  -0.4 
Instability  at  discharge  (percentage  points)  +11.31 

1,282 
+0.43 
+0.91 
+3.1 

996 
+0.33 
+1.4 
+2.9 

p  <0.01 

NS 
p  <0.10 

NOTES:  Table  entries  give  the  "raw"  differences  (post-PPS  minus  pre-PPS)  in  the  indicated 
variables  (adjustment  for  sickness  at  admission  had  little  effect).  NS  =  not  significant  at  the  10 
percent  level. 


In  contrast,  initial  use  of  DNR  and  instability  at  discharge  showed  consistent  increases 
post-PPS  across  the  three  hospital  types  (Table  8.4).  In  analyses  not  reported  here,  we  did 
find  some  differences  in  the  magnitude  of  the  pre-PPS  minus  post-PPS  increase  in  DNR 
status  as  a  function  of  patient  characteristics,  with  the  biggest  rise  post-PPS  concentrated  in 
patients  with  poor  function  and  high  levels  of  acute  sickness  at  admission.  Using  the  vari- 
ables mentioned  in  Methods,  above,  to  define  subgroups,  we  were  not  able  to  find  any  types  of 
patients  for  any  of  the  five  study  diseases  for  whom  the  rise  in  discharge  instability  post-PPS 
differed  significantly  from  the  average  increase  in  instability  for  all  patients  with  that 
disease. 
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Trend  Analyses 

Figure  8. 1  shows  that  the  quality  of  inhospital  process  of  care  was  on  the  rise  before  the 
introduction  of  PPS  (p  <  0.01)  and  continued  to  rise  post-PPS,  although  at  a  slightly  slower 
rate  (the  difference  between  the  rates  is  not  significant).  Fig.  8.2  shows  that  use  of  initial 
DNR  orders  was  increasing  in  1981-1982  (p  <  0.05)  and  that  this  increase  continued  in 
1985-1986.  The  percentage  of  patients  discharged  with  instability  was  reasonably  flat  both 
in  1981-1982  and  in  1985-1986,  with  a  jump  between  these  two  periods  (Fig.  8.3).  Sickness 
at  discharge  and  discharge  to  a  nursing  home  also  had  jumps  between  these  time  periods 
(not  shown).  Figure  8.4  presents  quarterly  values  of  180-day  mortality  aggregated  across  the 
five  diseases,  after  adjusting  for  sickness  at  admission.  There  was  no  trend  in  either  period, 
and  the  post-PPS  adjusted  mortality  values  were  consistent  with  the  pre-PPS  values. 


DISCUSSION 

After  adjusting  for  sickness  at  admission,  mortality  was  unchanged  or  lower  in 
1985-1986  after  the  introduction  of  PPS  than  in  1981-1982  before  PPS  was  implemented. 
We  believe  that  this  was  the  result  of  two  counterbalancing  forces:  sharp  improvements  in 
inhospital  process  from  1981  to  1986  that  acted  to  lower  mortality  rates,  offset  by  increases 
during  the  same  period  in  sickness  at  admission,  the  use  of  DNR  orders,  and  instability  at 
discharge  that  acted  to  raise  mortality.  However,  most  of  the  process  improvements  we  have 
documented  were  probably  not  caused  by  PPS.  Two  types  of  evidence  presented  here  support 
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Fig.  8.1 — Inhospital  process  trend  (five  diseases,  n  =  14,012) 


131 


10 


1980         1981         1982         1983         1984         1985         1986  1987 
Fig.  8.2— Initial  DNR  trend  (five  diseases,  n  =  14,012) 


this  conclusion:  the  presence  of  a  significant  upward  trend  in  process  in  the  pre-PPS 
1981—1982  period,  and  the  lack  of  uniformity  of  the  pre-  and  post-PPS  improvements  in  pro- 
cess across  different  types  of  patients  and  hospitals.  It  is  difficult  to  understand  why,  if  PPS 
caused  the  gains  in  process  we  have  demonstrated,  it  did  so  with  substantially  more  force  in 
rural  nonteaching  facilities  than  in  urban  teaching  hospitals.  Rather,  it  seems  more  plausi- 
ble that  ongoing  trends  in  medicine,  such  as  diffusion  of  newer  methods  into  outlying  areas, 
were  responsible  for  the  observed  improvement.  On  this  score  PPS  may  be  judged  a  success: 
Process  did  not  deteriorate  in  the  effort  to  save  money. 

An  increased  rate  of  DNR  orders  at  admission  post-PPS  was  associated  with  somewhat 
higher  death  rates.  DNR  orders  potentially  save  the  hospital  money  and  the  patients  pain, 
but  increased  use  of  them  is  associated  with  an  increase  in  death  rates.  The  patient  sub- 
groups in  which  the  biggest  changes  in  DNR  use  were  concentrated — those  who  were  both 
acutely  ill  and  functioning  poorly — suggest  that  the  potential  benefits  of  medical  care  are 
being  examined  more  closely  for  these  patients.  This  makes  sense  and  does  not  implicate 
PPS. 

Instability  at  discharge  and  related  factors  have  risen  in  a  way  that  is  temporally  asso- 
ciated with  PPS.  Furthermore,  instability  is  increasing  across  the  board  rather  than  in  any 
specific  patient  or  hospital  subgroup  and  has  increased  in  ways  that  we  believe  are  not  due  to 
changes  in  recording  (Sec.  VI).  Both  results  implicate  PPS.  On  the  face  of  it,  this  is  a  prob- 
lem that  needs  attention. 
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Fig.  8.3 — Instability  at  discharge  trend  (five  diseases,  n  =  7,412) 


However,  there  is  room  for  debate  about  how  serious  a  problem  this  is.  First,  there  is 
the  issue  of  whether  instability  should  be  denned  broadly  or  narrowly.  If  it  is  defined  nar- 
rowly to  include  only  factors  that  were  clearly  remediable,  its  increase  post-PPS  may  be 
responsible  for  0.5  percentage  points  of  mortality  increase,  depending  on  disease.  If  it  is 
defined  broadly  to  include  sickness  at  discharge  and  discharge  to  a  nursing  home,  the  associ- 
ated rise  in  death  rates  may  be  as  high  as  0.9  percentage  points.  We  do  not  know  how  fixable 
the  latter  group  of  problems  are,  but  we  do  know  that  we  measured  only  a  fraction  of  prob- 
lems that  should  be  fixed  before  discharge. 

Second,  there  is  no  guarantee  that  the  hospital  would  be  able  to  reduce  mortality  by 
holding  unstable  patients  longer.  Third,  the  effects  of  instability  might  be  reduced  by  more 
cost-effective  mechanisms  other  than  increasing  hospital  length  of  stay.  For  example, 
unstable  patients  could  be  given  special  consideration  in  nursing  homes  or  could  be  part  of 
well  designed  posthospitalization  home  nursing  programs.  Indeed,  the  mortality  impact  of 
unstable  discharge  does  not  appear  to  be  as  great  in  1985-1986  as  it  was  in  1981-1982. 
Fourth,  length  of  stay  has  remained  relatively  constant  since  1986,  so  that  any  contribution 
of  inappropriately  short  hospital  stays  to  discharge  problems  has  probably  stabilized 
(Witsberger  and  Kominsky,  1990). 

On  the  other  hand,  eliminating  unstable  discharges  entirely  (not  just  those  evident 
post-PPS)  might  have  a  large  impact  on  the  effectiveness  of  hospital  care.  For  the  five 
diseases  studied,  unstable  patients  in  the  post-PPS  period  have  a  mortality  rate  30  percent 
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Fig.  8.4 — Adjusted  180-day  mortality  trend  (five  diseases,  n  =  12,823) 


higher  than  stable  patients.  This  translates  into  4.4  percentage  points  of  additional  mortal- 
ity in  the  90  days  following  admission  for  patients  with  discharge  instability.  An  observa- 
tional study  such  as  ours  cannot  definitively  estimate  the  effects  of  better  discharge  monitor- 
ing on  patient  outcomes;  a  controlled  experiment  with  specific  discharge  protocols  is 
required. 

Between  1981-1982  and  1985—1986  there  were  also  important  changes  in  the  demo- 
graphics and  sickness  of  patients.  Pneumonia  patients  are  much  sicker  at  admission  now 
than  before.  For  other  diseases,  the  nature  of  the  change  in  admission  sickness  is  less  clear. 
As  we  mentioned  in  Sec.  II,  some  of  the  apparent  increase  in  sickness  appears  to  be  a  record- 
ing bias.  When  we  move  from  recording-sensitive  sickness  indicators  (such  as  the  functional 
status  and  acute  and  chronic  sickness  measures  mentioned  above  and  in  Table  8.2)  to  the 
recording-insensitive  indicators  used  in  the  remainder  of  this  study's  analyses  (such  as  our 
30-day  and  180-day  scales),  the  apparent  rise  in  admission  sickness  is  smaller.  Another  part 
of  the  rise  in  severity  may  be  due  to  efforts  by  the  PROs  to  restrict  admissions  of  less  sick 
patients.  For  two  diseases  (acute  myocardial  infarction  and  pneumonia)  admissions  of  the 
least  sick  patients  appear  to  be  down  about  30  percent. 

In  conclusion,  because  PPS  was  not  introduced  as  an  experiment,  our  observational 
time  series  study  can  provide  only  limited  answers  about  the  changes  in  quality  of  care  that 
PPS,  and  PPS  alone,  caused.  As  we  noted  in  Sec.  VII  three  caveats  are  also  worth  bearing  in 
mind  when  considering  our  results:  (1)  We  studied  only  five  diseases,  albeit  five  important 
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diseases  in  the  Medicare  cohort;  (2)  our  study  design  allowed  us  to  assess  differences  in  qual- 
ity of  care  only  after  patients  had  been  hospitalized,  so  that,  for  example,  we  cannot  comment 
on  any  changes  in  access  to  hospital  care  that  PPS  may  have  caused;  and  (3)  our  study  covers 
the  post-PPS  era  only  through  July  1986.  Since  that  time  hospital  payments  under  PPS  have 
been  tightened  (ProPac,  1988). 

Even  so,  two  key  policy  conclusions  appear  clear:  (1)  At  least  through  the  middle  of 
1986,  PPS  did  not  interrupt  an  important  long-term  trend  toward  better  processes  of  inhospi- 
tal  care,  a  trend  that  has  led  to  somewhat  lower  death  rates;  (2)  on  the  other  hand,  we  believe 
that  PPS  has  had  a  notable  effect  on  the  stability  with  which  patients  are  discharged.  These 
conclusions  lead  to  three  major  policy  recommendations:  (1)  To  eliminate  any  possible  prob- 
lems with  patient  instability  at  discharge,  a  more  systematic  assessment  of  the  readiness  of  a 
patient  to  leave  the  hospital  and  be  cared  for  as  an  outpatient  or  in  an  institution  other  than 
an  acute  care  hospital  should  be  made.  Perhaps  our  instability-at-discharge  scale  would  be  a 
good  place  to  start.  (2)  If  further  investigation  suggests  that  some  unstable  discharges  may 
be  acceptable  with  suitable  follow-up  or  could  be  prevented  by  longer  hospital  stays,  we 
recommend  that  clinical  trials  be  undertaken  to  evaluate  the  impact  of  such  changes  on  mor- 
tality. (3)  To  provide  current  information  about  the  effects  of  Medicare's  payment  methods 
on  quality  of  care,  we  also  recommend  the  continued  collection  of  clinically  detailed  data  that 
regularly  monitor  sickness  at  admission,  processes  of  care,  discharge  status,  and  outcomes  as 
long  as  prospective  payment  is  in  place. 


Appendix  A 


FURTHER  DETAILS  ON  DESIGN,  SAMPLING, 
AND  FIELDWORK 


SAMPLING  DISEASES 

We  selected  a  disease-specific  model  for  measuring  quality  of  care.  We  chose  this 
approach  because  we  believed  it  would  allow  us  to  collect  more  clinically  meaningful  data 
about  patients  for  quality  of  care  analyses. 

We  included  only  diseases  that  were  generally  stable  diagnostically  and  therapeutically, 
had  well-defined  clinical  criteria  for  making  the  diagnosis,  and  could  be  identified  with  rea- 
sonable certainty  in  both  time  periods  using  ICD-9-CM  codes  from  1981  to  1986. 

We  sampled  patients  with  each  of  the  six  study  diseases  by  ICD-9-CM  codes.  Table  A.l 
lists  the  six  diseases  by  the  ICD-9-CM  code  that  we  believed  encompassed  all  possible 
definitions  of  the  diseases  and  also  lists  the  corresponding  DRGs.  Approximately  19  percent 
of  Medicare  hospital  admissions  and  over  30  percent  of  30-day  mortality  are  for  patients  with 
one  of  these  six  diseases  (HCFA,  1986a). 

SAMPLING  PATIENTS 

Our  goal  was  to  sample  roughly  equal  numbers  of  patients  in  each  disease  at  each  hos- 
pital. The  national  distribution  of  patients  by  disease  per  hospital  makes  this  feasible 
without  special  adjustments  for  all  the  diseases  we  studied  except  depression,  which 
presented  unique  challenges  for  the  following  reasons. 

•  Some  hospitals  have  special  psychiatric  units  in  which  patients  with  depression  may 
be  treated;  some  do  not. 

•  Patients  at  a  hospital  with  such  a  unit  may  be  treated  either  in  the  psychiatric  unit 
or  in  one  of  the  scatterbeds  in  the  rest  of  the  hospital. 

•  In  the  post-PPS  period,  the  psychiatric  units  in  some  hospitals  are  exempt  from 
PPS;  other  such  units  fall  under  PPS  reimbursement. 

To  have  a  chance  to  estimate  the  causal  effect  of  these  structural  differences  on  quality  of 
care,  we  had  to  adopt  a  somewhat  complicated  design  that  attempted  to  insure  as  much  vari- 
ation across  these  factors  in  the  pre-PPS  and  post-PPS  time  periods  as  possible.  Table  A.2 
summarizes  the  design  goals  we  settled  on,  after  examining  MedPAR  patient  prevalence  data 
to  see  what  was  feasible  in  the  chosen  states  and  areas.  The  entries  in  the  table  show  the 
number  of  patients  in  each  cell  that  would  result  in  the  desired  sampling  plan. 

As  a  result  of  the  structural  factors  just  mentioned,  the  distribution  of  depression  patients 
is  highly  irregular  across  short-term  general  care  hospitals:  none  or  very  few  at  some  facilities 
and  a  large  number  at  others.  For  this  reason,  depression  ended  up  driving  the  sampling  plan 
to  a  larger  extent  than  any  of  the  other  diseases.  We  attempted  to  insure  that  an  adequate 
number  of  depression  records  would  be  available  from  each  state  by  using  patient  prevalences 
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Table  A.l 

SIX  DISEASES  DEFINED  BY  ICD-9-CM  CODES  AND  CATEGORIZED  BY  DRGS 


Principal  Relative 

DRG  DRG  ICD-9-CM 

Diagnosis  DRG        Weight  ICD-9-CM  Diagnosis 


Congestive  Heart  Failure 


HEART  FAILURE 
&  SHOCK 

CIRCULATORY  DISORDERS 
EXC  AMI ,   W  CARD  CATH 
&  COMPLEX  DIAG 

CIRCULATORY  DISORDERS 
EXC  AMI,   W  CARD  CATH 
W/O  COMPLEX  DIAG 


127         1.0164  428.0 

428. 1 

124        2.1680  428.0 

428. 1 


125         1.6069  428.0 

428.1 


Congestive  Heart  Failure 
Left  Heart  Failure 

Congestive  Heart  Failure 
Left  Heart  Failure 


Congestive  Heart  Failure 
Left  Heart  Failure 


ACUTE  MYOCARDIAL  INFARCTION 


PERMANENT  CARDIAC 
PACEMAKER  IMPLANT 
WITH  AMI  OR  CHF 


115 


3.8232 


410 

410, 

410 

410, 

410, 

410 

410, 

410, 

410, 

410, 


AMI  Anterolateral  Wall 
AMI  Anterior  Wall  Nec 
AMI  Inferolateral  Wall 
AMI  Inf eroposterior  Wall 
AMI  Inferior  Wall  Nec 
AMI  Lateral  Wall  Nec 
True  Posterior  Infarct 
Subendocardial  Infarct 
Myocardial  Infarct  Nec 
Myocardial  Infarct  Nos 


CIRCULATORY  DISORDERS 
W  AMI  &  C.V.  COMP. 
DISCH.  ALIVE 


121 


1.8211 


410 
410 
410 
410 
410 
410 
410 
410 
410 
410 


AMI  Anterolateral  Wall 
AMI  Anterior  Wall  Nec 
AMI  Inferolateral  Wall 
AMI  Inf eroposterior  Wall 
AMI  Inferior  Wall  Nec 
AMI  Lateral  Wall  Nec 
True  Posterior  Infarct 
Subendocardial  Infarct 
Myocardial  Infarct  Nec 
Myocardial  Infarct  Nos 
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Table  A.l — continued 


Principal 

Relative 

DRG 

DRG 

ICD-9-CM 

Diagnosis 

DRG  Weight 

ICD-9-CM 

Diagnosis 

CIRCULATORY  DISORDERS 

122  1.3331 

410.0 

AMI  Anterolateral  Wall 

W  AMI  W/0  C.V.  COMP. 

410.1 

AMI  Anterior  Wall  Nec 

DISCH.  ALIVE 

410.2 

AMI  Inferolateral  Wall 

410  3 

AMT    Tnf prnnn^tprior  WaI 1 

«U  IX       X  1 1  J-  C  X  vUv  O  l«  w  J_  X      X       V*  Ca  X  X 

410.4 

AMI  Inferior  Wall  Nec 

410.5 

AMI  Lateral  Wall  Nec 

410.6 

True  Posterior  Infarct 

410.7 

Subendocardial  Infarct 

410.8 

Myocardial  Infarct  Nec 

410.9 

Myocardial  Infarct  Nos 

CIRCULATORY  DISORDERS 

123  1.1094 

410.0 

AMI  Anterolateral  Wall 

W  AMI,  EXPIRED 

410.1 

AMI  Anterior  Wall  Nec 

410.2 

AMI  Inferolateral  Wall 

410.3 

AMI  Inf eroposterior  Wall 

410.4 

AMI  Inferior  Wall  Nec 

410.5 

AMI  Lateral  Wall  Nec 

410.6 

True  Posterior  Infarct 

410.7 

Subendocardial  Infarct 

410.8 

Myocardial  Infarct  Nec 

410.9 

Myocardial  Infarct  Nos 

ACUTE  MYOCARDIAL 

INFARCTION 

CARDIAC  PACEMAKER 
REPLACE  &  REVIS  EXC 
PULSE  GEN  REPL  ONLY 


117  1.7783 


37.81  Replace  Transven  Electrod 

37.82  Replace  Epicard  Electrod 

37.83  Remove  Transven  Electrode 

37.84  Remove  Epicard  Electrode 
37.86  Remove  Pacemaker  System 
37.89  Revise  Pacemaker  Sys  Nec 


CARDIAC  PACEMAKER 
PULSE  GENERATOR 
REPLACEMENT  ONLY 


118  1.7392 


37.85 


Replace  Pulse  Generator 


CARDIAC  ARRHYTHMIA 

&  CONDUCTION  DISORDERS 

AGE  <7  0  W/O  CC 


139  .8108  426.0  Atriovent  Block  Complete 

426.10  Atriovent  Block  Nos 

426.11  Atriovent  Block-lst  Degree 

426.12  Atriovent  Block-Mobitz  II 

426.13  Av  Block-2nd  Degree  Nec 

426.2  Left  BB  Hemiblock 

426.3  Left  BB  Block  Nec 

426.4  Rt  Bundle  Branch  Block 
426.50  Bundle  Branch  Block  Nos 
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Table  A.l — continued 


Principal 

Relative 

DRG 

DRG 

ICD-9-CM 

Diagnosis 

DRG  Weight 

ICD-9-CM 

Diagnosis 

426.51 

Rt  BBB/Lft  Post  Fasc  Blk 

426.52 

Rt  BBB/Lft  Ant  Fasc  Blk 

426.53 

Bilat  BB  Block  Nec 

426.54 

Trifascicular  Block 

426.6 

Other  Heart  Block 

426.89 

Conduction  Disorder  Nec 

426.9 

Conduction  Disorder  Nos 

427.81 

Sinoatrial  Node  Dysfunct 

427.89 

Cardiac  Dysrhythmias  Nec 

427  .9 

Cardiac  Dysrhythmias  Nos 

746 . 86 

Congenital  Heart  Block 

ACUTE  MYOCARDIAL 

INFARCTION 

CARDIAC  ARRHYTHMIA 

138  .9079 

426  .7 

Anomalous  A  Excitation 

&  CONDUCTION  DISORDERS 

426.81 

Lown-Ganong-Levine  Synd 

AGE  >=7  0  &/OR  CC 

427.0 

Parox  Atrial  Tachycardia 

427.1 

Parox  Ventric  Tachycardi 

427.2 

Parox  Tachycardia  Nos 

427  .31 

Atrial  Fibrillation 

427.32 

Atrial  Flutter 

A  0  7    £  1 

ventricular  r  loniiation 

427.42 

Ventricular  Flutter 

427.60 

Premature  Beats  Nos 

427.61 

Atrial  Premature  Beats 

427.69 

Premature  Beats  Nec 

785.0 

Tachycardia  Nos 

785. 1 

Palpitations 

CARDIAC  ARRHYTHMIA 

139  .8108 

426.7 

Anomalous  A  Excitation 

&  CONDUCTION  DISORDERS 

426.81 

Lown-Ganong-Levine  Synd 

AGE  <7  0  W/O  CC 

427.0 

Parox  Atrial  Tachycardia 

427  .1 

Parox  Ventric  Tachycardi< 

427  .2 

Parox  Tachycardia  Nos 

427.31 

Atrial  Fibrillation 

427.32 

Atrial  Flutter 

427.41 

Ventricular  Fibrillation 

427.42 

Ventricular  Flutter 

427.60 

Premature  Beats  Nos 

427.61 

Atrial  Premature  Beats 

427.69 

Premature  Beats  Nec 

785.0 

Tachycardia  Nos 

785.1 

Palpitations 

139 


Table  A.l — continued 


Principal  Relative 

DRG  DRG  ICD-9-CM 

Diagnosis  DRG        Weight  ICD-9-CM  Diagnosis 


HIP  FRACTURE 


FRACTURES  OF  HIP  236        1.3530  820.00  Fx  femur  intrcaps  nos-cl 

&  PELVIS  820.02  Fx  femur  midcervic-clos 

820.03  Fx  base  femoral  nck-clos 

820.09  Fx  femur  intrcaps  nec-cl 

820.20  Trochanteric  fx  nos-clos 

820.21  Intertrochanteric  fx-cl 
820.80  Fx  neck  of  femur  nos-cl 


PNEUMONIA  (PNE) 


RESPIRATORY  INFECTION  79  1.7561 

&  INFLAMMATION 
AGE  >=7  0  &/OR  CC 


4  82.0  K.  Pneumoniae  Pneumonia 

482.1  Pseudomonal  Pneumonia 

482.4  Staphylococcal  Pneumonia 

484.8  Pneum  in  Infect  Dis  Nec 

507.0  Food/Vomit  Pneumonitis 


RESPIRATORY  INFECTION  80  1.7036 

&  INFLAMMATION 
AGE  18-69  W/O  CC 


4  82.0  K.  Pneumoniae  Pneumonia 

4  82.1  Pseudomonal  Pneumonia 

482.4  Staphylococcal  Pneumonia 

484.8  Pneum  in  Infect  Dis  Nec 

507.0  Food/Vomit  Pneumonitis 


SIMPLE  PNEUMONIA  89  1.0771 

&  PLEURISY 

AGE  >=70  &/OR  CC 


480.9  Viral  Pneumonia  Nos 

481.  Pneumococcal  Pneumonia 

482.2  H.   Influenzae  Pneumonia 

4  82.3  Stremtococcal  Pneumonia 

482.8  Bacterial  Pneumonia  Nec 

482.9  Bacterial  Pneumonia  Nos 
483.  Pneumonia:     Organism  Nec 

485.  Broncopneumonia  Org  Nos 

486.  Pneumonia,  Organism  Nos 
487.0  Influenza  with  Pneumonia 


SIMPLE  PNEUMONIA  90  .9618 

&  PLEURISY 

AGE  18-69  W/O  CC 


480.9  Viral  Pneumonia  Nos 

481.  Pneumococcal  Pneumonia 

482.2  H. Influenzae  Pneumonia 

4  82.3  Streptococcal  Pneumonia 

482.8  Bacterial  Pneumonia  Nec 

4  82.9  Bacterial  Pneumonia  Nos 
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Table  A. l--continued 


Principal  Relative 

DRG  DRG  ICD-9-CM 

Diagnosis  DRG        Weight  ICD-9-CM  Diagnosis 


483  . 

Pneumonia:     Organism  Nec 

485. 

Broncopneumonia  Org  Nos 

486  . 

Pneumonia,  Organism  Nos 

487.0 

Influenza  with  Pneumonia 

511.0 

Pleurisy  W/O  Effus  or  TB 

PNEUMONIA 

BRONCHITIS  6 

.7809 

464.10 

AC  Tracheitis  No  Obstruc 

&  ASTHMA  AGE 

464.11 

AC  Tracheitis  W  Obstruct 

>=7  0   &/OR  CC 

A  C  C  A 

4  o  o  .  U 

Acute  Bronchitis 

466.1 

Acute  Bronchiolitis 

490. 

Bronchitis  Nos 

491 .  0 

Simple  Chr  Bronchitis 

493.00 

Ext  Asthma  W/O  Stat  Asth 

493.01 

Ext  Asthma  W/O  Stat  Asth 

/iQ'i  in 

4  j  J  •  JL  U 

Tnf    Jc+hma    W/C>    Q-l-a'l-  ftal-Vi 
Xilu    rioT^niTla    W  /  \J                  rib  cn 

493.90 

Asthma  W/O  Status  Asthm 

493.91 

Asthma  W  Status  Asthmat 

519.1 

Trachea/Bronchus  Dis  Nec 

HEART  FAILURE                           12  7 

1 .0164 

398.91 

Rheumatic  Heart  Failure 

&  SHOCK 

402.01 

Mai  Hypert  Hrt  Dis  W  CHF 

402  .  11 

Benign  Hyp  Hrt  Dis  W  CHF 

402.91 

Hyperten  Heart  Dis  W  CHF 

428.9 

Heart  Failure  Nos 

785.50 

Shock  Nos 

785.51 

Cardiogenic  Shock 

785.59 

Shock  W/O  Trauma  Nec 

PULMONARY  EDEMA  87 

1.5165 

506.1 

Fum/Vapaor  Ac  Pum  Edema 

&  RESPIRATORY  FAILURE 

514. 

Pulm  Congest/Hypostasis 

518.4 

Acute  Lung  Edema 

518.5 

Post  Traum  Pulm  Insuffic 

799  .1 

Respiratory  Failure 

ATHEROSCLEROSIS  132 

.8967 

412  . 

Old  Myocardial  Infarct 

AGE  >=7  0  &/OR  CC 

414.0 

Coronary  Atherosclerosis 

414.8 

Chr  Ischemic  Hrt  Dis  Nec 

414.9 

Chr  Ischemic  Hrt  Dis  Nos 

429  .2 

Ascvd 

429.3 

Cardiomegaly 

429.89 

Ill-Defined  Hrt  Dis  Nec 

429  .9 

Heart  Disease  Nos 

459.89 

Circulatory  Disease  Nec 

459.9 

Circulatory  Disease  Nos 

793.2 

ABN  FD-Intrathor  Org  Nec 
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Table  A.l — continued 


Principal  Relative 

DRG  DRG  ICD-9-CM 

Diagnosis  DRG        Weight  ICD-9-CM  Diagnosis 


CEREBROVASCULAR  DISEASE 


SPECIFIC  CEREBROVASCULAR 

14 

1 .3210 

430. 

Subarachnoid  hemorrhage 

DISORDERS  EXCEPT  TIA 

431. 

Intracerebral  hemorrhage 

432.0 

Nontraum  extradural  hem 

432.9 

Intracranial  hemorr  nos 

434  .  0 

Cerebral  thrombosis 

434.1 

Cerebral  embolism 

434.9 

Cerebr  artery  occlus  nos 

436. 

CVA 

437.3 

Nonrupt  cerebral  aneurym 

784  .3 

Aphasia 

SPECIFIC  CEREBROVASCULAR 

14 

1 .3210 

430. 

Subarachnoid  hemorrhage 

DISORDERS  EXCEPT  TIA 

431. 

Intracerebral  hemorrhage 

432  .0 

Nontraum  extradural  hem 

432  .9 

Intracranial  hemorr  nos 

434  .  0 

Cerebral  thrombosis 

434  .1 

Cerebral  embolism 

434  .9 

Cerebr  artery  occlus  nos 

436  . 

CVA 

437  .3 

Nonrupt  cerebral  aneurym 

784  .3 

Aphasia 

DEPRESSION 

DEPRESSIVE  NEUROSES 

426 

.9272 

300.4 

Neurotic  Depression 

(MILDER  DEPRESSION) 

301.12 

Chr  Depressive  Person 

309.0 

Brief  Depressive  React 

309.1 

Prolong  Depressive  React 

311 . 

Depressive  Disorder  Nec 

300.9 

Neurotic  Disorder  Nos 

308.0 

Stress  React,  Emotional 

308  .1 

Stress  Reaction,  Fugue 

308.2 

Stress  React,  Psychomot 

308.3 

Acute  Stress  React  Nec 

308.4 

Stress  React,  Mixed  Dis 

308.9 

Acute  Stress  React  Nos 

780.1 

Hallucinations 

799.2 

Nervousness 
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Table  A.l — continued 


Principal 

Relative 

DRG 

DRG 

ICD-9-CM 

Diagnosis 

DRG 

Weight 

ICD 

-9-CM 

Diagnosis 

DEPRESSION 

PSYCHOSIS 

430 

1  .0678 

296 

.15 

Recur  Manic-Part  Remiss 

296 

.16 

Recur  Manic-Full  Remiss 

296 

.20 

Depress  Psychosis-Unspec 

296 

.21 

Depress  Psychosis-Mild 

296 

.22 

Depressive  Psychosis-Mod 

296 

.23 

Depress  Psychosis-Severe 

296 

.24 

Depr  Psychos-Sev  W  Psych 

296 

.25 

Depr  Psychos-Part  Remiss 

296 

.26 

Depr  Psychos-Full  Remiss 

296 

.30 

Recurr  Depr  Psychos-Unsp 

296 

.31 

Recurr  Depr  Psychos-Mild 

296 

.32 

Recurr  Depr  Psychos-Mod 

296 

.33 

Recur  Depr  Psych-Severe 

296 

.34 

Rec  Depr  Psych-Psychotic 

296 

.35 

Recur  Depr  Psyc-Part  Rm 

296.36  Recur  Depr  Psych-Full  Rem 

296.40  Bipol  Aff,  Manic-Unspec 

296.41  Bipolar  Aff,  Manic-Mild 

296.42  Bipolar  Affec,  Manic-Mod 

296.43  Bipol  Aff,  Manic-Severe 

296.44  Bipol  Manic-Sev  W  Psych 

296.45  Bipol  Aff  Manic-Part  Rem 

296.46  Bipol  Aff  Manic-Full  Rem 

296.50  Bipolar  Aff,  Depr-Unspec 

296.51  Bipolar  Affec,  Depr-Mild 

296.52  Bipolar  Affec,  Depr-Mod 

296.53  Bipol  Aff,  Depr-Severe 

296.54  Bipol  Depr-Sev  W  Psych 

296.55  Bipol  Aff  Depr-Part  Rem 

296.56  Bipol  Aff  Depr-Full  Rem 

296.60  Bipol  Aff,  Mixed-Unspec 

296.61  Bipol  Aff,  Mixed-Mild 

296.62  Bipolar  Affec,  Mixed-Mod 

296.63  Bipol  Aff,  Mixed-Severe 

296.64  Bipol  Mixed-Sev  Ww  Psych 

296.65  Bipol  Aff,  Mix-Part  Rem 

296.66  Bipol  Aff,  Mix-Full  Rem 
296.7  Bipolar  Affective  Nos 
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Table  A.l — continued 


Principal 
DRG 

Diagnosis 


DRG 


Relative 
DRG 

Weight 


ICD-9-CM 


ICD-9-CM 
Diagnosis 


PSYCHOSES 
(continued) 


43- 


1.0678 


OTHER  DIAGNOSES 

OF  MENTAL  DISORDERS 


432 


1.0278 


296 
296 
296 
296 
296 
296 
297 
297 
297 
297 
297 
297 
298 
298 
298 
298 
298 
298 


80 

81 

82 

89 

90 

99 

0 

1 

2 

3 

8 

9 

0 

1 

3 

4 

8 

9 


298.2 
302.0 
302  . 1 
302  .2 
302  .3 


Manic-Depressive  Nos 
Atypical  Manic  Disorder 
Atypical  Depressive  Dis 
Manic-Depressive  Nec 
Affective  Psychosis  Nos 
Affective  Psychoses  Nec 
Paranoid  State,  Simple 
Paranoia 
Paraphrenia 

Shared  Paranoid  Disorder 
Paranoid  States  Nec 
Paranoid  State  Nos 
React  Depress  Psychosis 
Excitativ  Type  Psychosis 
Acute  Paranoid  Reaction 
Psychogen  Paranoid  Psych 
React  Psychosis  Nec/Nos 
Psychosis  Nos 

Reactive  Confusion 

Homosexuality 

Zoophilia 

Pedophilia 

Transvestism 


DEPRESSION 


OTHER  DIAGNOSES  OF  432  1.0278 

MENTAL  DISORDERS 

(Continued) 


302 

.4 

Exhibitionism 

302 

.50 

Trans-Sexualism  Nos 

302 

.51 

Trans-Sexualism,  Asexual 

302 

.52 

Trans-Sexual,  Homosexual 

302 

.53 

Trans-Sex,  Heterosexual 

302 

.6 

Psychosex  Identity  Dis 

302 

.70 

Psychosexual  Dysfunc  Nos 

302 

.71 

Inhibited  Sexual  Desire 

302 

.72 

Inhibited  Sex  Excitement 

302 

.73 

Inhibited  Female  Orgasm 

302 

.74 

Inhibited  Male  Orgasm 

302 

.75 

Premature  Ejaculation 

302 

.76 

Functional  Dyspareunia 

302 

.79 

Psychosexual  Dysfunc  Nec 

302 

.81 

Fetishism 

302 

.82 

Voyeurism 
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Table 

A. 1 — continued 

Principal 

Relative 

DRG 

DRG 

ICD-9-CM 

Diagnosis 

DRG 

Weight 

ICD-9-CM 

Diagnosis 

Sexual  Masochism 

J  U  Z  .  OH 

Sexual  Sadism 

O  A  O     O  C 

Gender  Ident  Dis,  Adult 

O  A  o  OA 

joz . 

Psychosexual  Dis  Nec 

"3  A  O  Q 

Psychosexual  Dis  Nos 

o  a  £  o 
306  .  8 

Psychogenic  Disorder  Nec 

O  A  "7  A 

.307.0 

Stammering,  Stuttering 

307  .3 

Stereotyped  Movements 

O  A  "7     ^  A 

o  U  /  .  4  U 

Nonorganic  Sleep  Dis  Nos 

*3  a  n     a  1 

J  (J  /  .  4  1 

Transient  Insomnia 

*3  A  "7      d  O 

J  U  /  .  4  Z 

Persistent  Insomnia 

307.43 

Transient  Hypersomnia 

307.44 

Persistent  Hypersomnia 

- 

307.45 

Disrupt  Sleep-Wake  Cycle 

307  .46 

Somnambulism/Night  Terro] 

307.47 

Sleep  Stage  Dysfunc  Nec 

307.48 

Repetit  Sleep  Intrusion 

307.49 

Nonorganic  Sleep  Dis  Nec 

307.50 

Eating  Disorder  Nos 

DEPRESSION 

OTHER  DIAGNOSES  OF 

432 

1.0278 

307.51 

Bulimia 

MENTAL  DISORDERS 

307.59 

Eating  Disorder  Nos 

780.50 

Sleep  Disturbance  Nos 

780.52 

Insomnia  Nec 

780.54 

Hypersomnia  Nec 

780.55 

Irreg  Sleep-Wake  Rhy  Nos 

780.56 

Sleep  Stage  Dysfunctions 

780.59 

Sleep  Disturbances  Nec 

784 .60 

Symbolic  Dysfunction  Nos 

ORGANIC  DISTURBANCES 

429 

.9300 

299.00 

Infantile  Autism-Active 

&  MENTAL  RETARDATION 

299.01 

Infantile  Autism-Resid 

317. 

Mild  Mental  Retardation 

318.0 

Mod  Mental  Retardation 

318.1 

Severe  Mental  Retard 

318.2 

Profound  Mental  Retard 

319. 

Mental  Retardation  Nos 

758.0 

Down ' s  Syndrome 

758.1 

Patau's  Syndrome 

758.2 

Edwards '  Syndrome 
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Table  A. 2 
SAMPLING  GOALS  FOR  DEPRESSION 

Pre-PPS 


Fee  for 

PPS  Service 

Hospital 

Status             Unit      Scatterbed        Unit      Scatterbed  Total 
+  +  +  +  .  + 

Exempt  unit     |      0       |             0           |     367      |           183         |  550 
+  +  .  +_  +  + 

Nonexempt  unit   |     0       |            0           |     267     |           133         |  400 
+  +  +  .  +  + 

Scatterbed       |     0       |            0          |         0     |          466         |  466 
+  +  +__  +  + 

Total  0  0  634  782  1416 

Post-PPS 

Fee  for 

PPS  Service 

Hospital 

Status              Unit      Scatterbed        Unit      Scatterbed  Total 
+  +  .  +  +  + 

Exempt  unit     |       0     |          183         |     367     |            0  |  550 

+  +  +  +  + 

Nonexempt  unit   |267|           133         |         0     |             0  |  400 

+  +  +  +  + 

Scatterbed       |     0       |  466         |         0     |  0  |  466 

+  +  +  +  + 

Total  267  782  367  0  1416 


from  HCFA's  1984  MedPAR  file  to  identify  21  hospitals  with  abundant  depression  records  in 
categories  where  there  are  not  many  patients  (e.g.,  the  nonexempt  unit  and  scatterbed  rows) 
and  including  them  in  the  sample.  In  effect  this  amounts  to  stratifying  on  "big  depression" 
rather  than  "small  depression"  hospitals  and  oversampling  the  "big  depression"  stratum. 

SAMPLING  STUDY  STATES 

We  used  a  multistage  cluster  sample  with  stratification,  probability  proportional  to  size 
allocation,  and  subsampling  with  units  of  unequal  size  for  hospital  and  patient  sampling.  We 
chose  five  states  in  which  PPS  was  employed  and  which  had  a  rich  diversity  of  hospital  and 
patient  types.  We  were  unable  to  include  a  large  northeastern  state  since  Massachusetts, 
New  York,  and  New  Jersey  used  alternative  reimbursement  systems  during  the  PPS  study 
period.  Within  each  state  we  collaborated  with  the  PRO  for  the  study. 

Figure  A.l  shows  the  states  selected. 
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INSTRUMENT  DEVELOPMENT  AND  DATA  COLLECTOR  PERFORMANCE 

To  measure  quality  of  care  we  performed  a  detailed  review  of  the  medical  record  and 
used  secondary  databases  to  collect  information  about  postdischarge  outcomes.  To  collect 
data  from  the  medical  record  we  developed  disease-specific  abstraction  forms,  which  guided 
the  collection  of  data  about  clinical  outcomes  including  death,  clinical  diagnoses,  unstable 
complexes  of  symptoms  and  signs,  and  abnormal  laboratory  values  (Kahn  et  al.,  1988b; 
Kosecoff  et  al.,  1988;  Sherwood  et  al.,  1988;  Roth  et  al.,  1988;  Rubenstein  et  al.,  1988;  and 
Wells  et  al.,  1988).  We  also  collected  detailed  information  about  patient  sickness  at  admis- 
sion to  compare  clinical  outcomes  after  adjustment  for  the  conditions  that  were  present  at 
admission.  In  addition,  we  collected  data  about  process  measures  using  clinically  detailed 
criteria.  Some  process  criteria  were  applied  to  all  patients;  others  were  applied  only  to 
patients  with  a  specific  clinical  condition.  For  example,  we  collected  information  about  the 
completeness  of  a  fever  work-up  only  for  patients  with  a  temperature  at  or  above  a  given 
level. 


Data  Collector  Monitoring 

During  the  period  of  data  collection,  the  research  team  visited  data  collectors  in  the  field 
at  least  once.  A  supervisory  project  manager  spoke  with  the  research  team  in  each  state  at 
least  weekly.  In  addition,  data  collectors  and  project  managers  were  invited  to  call  the 
RAND  telephone  hotline  and  speak  with  a  physician  about  any  questions  that  might  arise  in 
the  field  as  they  used  the  abstraction  form  to  abstract  medical  records.  Some  phone  calls 
were  used  to  clarify  the  meaning  of  an  item  in  the  abstraction  form  or  guidelines  for  a  chal- 
lenging patient  problem.  Other  phone  calls  were  used  to  discuss  the  best  means  for  abstract- 
ing data  given  a  patient  situation  that  had  not  been  previously  anticipated. 

The  Abstraction  Forms 

The  abstraction  forms  are  divided  into  14  sections: 

1.  Tear  sheet  for  confidential  patient  demographics 

2.  Identification  data  noting  patient,  hospital,  and  abstractor  code 

3.  Inclusion/exclusion  criteria 

4.  Patient  characteristics  describing  patient  demographics 

5.  General  medical  history 

6.  Physician  care 

7.  Nursing  care,  cardiac  monitoring,  and  vital  signs 

8.  Medications 

9.  X-rays 

10.  Electrocardiograms  (EKGs) 

11.  Laboratory  data 

12.  Special  tests  and  procedures 

13.  Admission/discharge  status 

14.  Completeness  check 

The  abstraction  form  was  designed  to  follow  the  format  of  a  medical  record  so  that 
abstraction  would  be  both  accurate  and  efficient.  For  each  item  or  group  of  items,  the  section 
of  the  medical  record  that  was  to  serve  as  the  source  of  data  was  specified.  For  example,  both 
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physicians'  and  nurses'  notes  were  sources  of  data  for  patient  symptoms  of  shortness  of 
breath.  In  addition,  items  derived  from  a  particular  portion  of  the  medical  record  were 
grouped.  Items  likely  to  be  found  in  the  physician's  notes  were  grouped  in  the  physician  care 
section  of  the  abstraction  form.  Items  likely  to  be  found  in  the  nurses'  notes,  or  in  documen- 
tation of  cardiac  monitoring  and  vital  signs,  were  grouped  in  the  nursing  care  section  of  the 
abstraction  form.  Data  about  use  of  medications  were  grouped  in  the  medications  section  of 
the  abstraction  form.  Data  about  X-rays,  electrocardiograms,  and  laboratory  studies  were 
also  grouped. 

The  pages  of  the  abstraction  form  were  color-coded  by  section  to  facilitate  the  location  of 
the  various  sections  as  new  data  from  the  medical  record  were  obtained. 

Inclusion  and  Exclusion  Criteria.  Patients  were  included  in  the  sample  if  they  met 
three  criteria.  First,  patients  had  to  have  been  hospitalized  within  the  time  frame  of  the 
study:  calendar  year  1981,  calendar  year  1982,  or  July  1,  1985,  through  June  30,  1986. 
Second,  the  medical  record  itself  was  reviewed  for  ICD-9-CM  codes.  If  the  record  had  the 
specified  ICD-9-CM  code,  or  had  no  ICD-9-CM  code  listed  (despite  the  hospital  having  a 
record  of  hospitalization  with  that  code),  the  patient  was  considered  eligible  for  inclusion. 
Third,  the  medical  record  was  reviewed  to  determine  if  the  actual  clinical  reason  for  hospital- 
ization was  as  it  should  be  for  the  sampled  disease.  For  each  disease,  exclusion  criteria  were 
also  reviewed  (see  below). 

Demographics  and  Complexity.  Data  about  demographics  and  patient  complexity 
were  collected  for  patients  selected  for  inclusion  in  the  study  based  upon  the  criteria  listed 
above.  Complexity  was  defined  to  measure  patient  sickness  at  admission  based  on  the  reason 
for  the  admission  (morbid  conditions)  and  on  other  conditions  that  might  influence  outcome 
or  process  (comorbid  conditions).  Both  acute  and  chronic  conditions  were  included  as  com- 
plexity measures.  Efforts  were  made  to  collect  data  about  complexity  as  close  to  admission  as 
possible  so  that  patient  status  at  admission  could  be  distinguished  from  patient  status  after 
the  delivery  of  inhospital  processes.  Complexity  data  were  collected  in  the  patient  informa- 
tion, general  medical  history,  nursing  care,  and  admission/discharge  status  sections  of  the 
abstraction  form. 

Process.  The  abstraction  form  is  a  guide  for  the  collection  of  data  about  process,  i.e., 
what  is  done  to  patients  during  the  hospital  stay.  Data  about  both  diagnostic  assessments 
and  therapeutics  were  collected  by  studying  physician  care  and  nursing  care.  Some  process 
measures  were  collected  for  all  patients;  other  process  measures  were  collected  only  for  sub- 
sets of  patients  for  whom  the  process  was  relevant.  Data  about  patient  sickness  during  the 
hospital  stay  were  collected  to  assure  that  process  criteria  could  be  matched  to  the  patients' 
clinical  needs. 

Outcome.  The  abstraction  form  is  also  a  guide  for  the  collection  of  data  about  out- 
comes, i.e.,  what  happens  to  patients  during  the  hospital  stay.  Data  were  collected  about 
death,  clinical  diagnoses,  symptoms,  signs,  chest  X-ray  reports,  and  laboratory  results. 
These  data  are  collected  from  the  physician  care,  chest  X-ray,  and  laboratory  sections  of  the 
abstraction  form. 

Patient  Status  at  Discharge.  Patient  status  at  discharge  is  defined  as  the  patient's 
clinical  condition  the  day  before  and  day  of  discharge.  Data  about  the  patient  during  this 
time  period  are  collected  from  the  physician  care,  nursing  care  (including  vital  signs  and  car- 
diac monitoring),  chest  X-ray,  and  laboratory  sections  of  the  abstraction  form. 
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Guidelines  for  the  Abstraction  Form 

To  standardize  the  abstraction  process,  a  detailed  set  of  guidelines  was  prepared  to 
accompany  the  abstraction  form.  The  guidelines  define  medical  terms,  specify  data  sources 
from  the  medical  record,  and  provide  important  medical  synonyms.  Each  item  in  the 
abstraction  form  has  a  corresponding  section  in  the  guidelines.  Despite  the  extensive  guide- 
lines, the  field  experience  presented  a  number  of  unique  patient  problems.  These  were 
identified  by  data  collectors  in  the  field,  supervisory  project  managers,  or  RAND  researchers. 
To  keep  the  data  collectors  aware  of  additions  or  changes  in  the  guidelines,  bimonthly 
updates  were  sent  to  each  data  collector  during  the  abstraction  process.  These  updates  have 
been  incorporated  into  the  guidelines  presented  in  Kahn  et  al.  (1988b);  Kosecoff  et  al.  (1988); 
Roth  et  al.  (1988);  Rubenstein  et  al.  (1988);  Sherwood  et  al.  (1988);  and  Wells  et  al.  (1988). 

DISEASE-SPECIFIC  INCLUSION  AND  EXCLUSION  CRITERIA 

We  used  multiple  criteria  to  assure  a  homogeneous  group  of  patients  with  specified 
diseases  in  the  pre-  and  post-PPS  periods.  First,  we  asked  hospitals  to  provide  a  list  of 
patients  with  our  ICD-9-CM  codes.  The  list  was  designed  to  include  enough  patients  to  allow 
an  average  of  about  9.5  patients  from  each  hospital  with  each  disease  to  be  included  in  the 
sample  even  though  some  records  would  be  unavailable  for  review  and  some  records  would  be 
inappropriate  for  the  sample  because  of  clinical  considerations. 

Second,  we  used  clinical  criteria  to  verify  that  the  patient  was  at  least  65  years  of  age, 
was  admitted  during  a  study  year,  and  was  hospitalized  for  the  study  disease.  Patients  with 
the  study  disease  as  a  complication  of  hospitalization,  rather  than  as  a  reason  for  admission, 
were  not  eligible  for  the  sample.  Except  for  depression,  we  did  not  include  patients  who  were 
transferred  from  another  acute  care  general  hospital  because  we  did  not  have  access  to  the 
medical  record  from  the  initial  hospital  stay  for  those  patients. 

Third,  we  used  disease-specific  exclusion  criteria  to  increase  the  likelihood  of  a  homoge- 
neous study  sample  across  both  time  periods.  Specifically,  we  excluded  patients  with  major 
competing  conditions  (e.g.,  end  stage  cancer  in  a  patient  with  congestive  heart  failure)  that 
were  likely  to  determine  process  and  outcome  for  the  patient  regardless  of  the  presence  of  the 
study  disease  on  admission. 

The  following  subsection  lists  the  clinical  inclusion  and  exclusion  criteria  for  each  study 
disease  (except  depression).  Table  A. 3  summarizes  the  clinical  exclusion  criteria  for  each 
disease  and  allows  the  comparison  of  exclusion  criteria  across  diseases. 

Congestive  Heart  Failure  Inclusion  Criteria 

1.    Medical  record  for  this  admission  shows  any  of  the  following  ICD-9-CM  codes  for 
CHF: 

428.XX 

X  is  any  value  0-9  or  no  value. 

and  2.    Patient  has  CHF  as  reason  for  admission. 

To  be  included  patients  must  have  at  least  one  from  (A)  and  at  least  one  from  (B) 
present  at  the  time  of  hospital  admission. 

(A)    Current  complaints  or  history  of: 
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Table  A. 3 


CLINICAL  EXCLUSION  CRITERIA  FOR  STUDY  SAMPLE 


Criterion 


CHF 


AMI 


PNE 


CVA  HIP 


General 

No  eligible  ICD-9  codes  X 
Age  <  65  years  X 
Transfer  from  another  in-patient  X 
service 

Not  hospitalized  in  specified  X 

study  year 
Specified  condition  not  the  X 

reason  for  admission 


X 
X 
X 

X 

X 


X 
X 
X 

X 

X 


X 
X 
X 

X 

X 


X 
X 
X 

X 

X 


Any  of  the  following  conditions 
Cardiovascular 
Acute  MI 

Cardiac  arrest /sudden  death 


X(l) 
X(2) 


X(l) 
X(2,3) 


X(2) 
X<2,3) 


Pulmonary 

Respiratory  arrest /sudden  death 
Lung  cancer   (primary  or 

metastases) 
Interstitial  lung  disease  at 

admission 
Active  TB  at  admission 


X(2) 


X(2) 
X 

X 

X 


X(2,3) 


Neurologic 
Acute  CVA 
Coma  at  admission 
Multiple  sclerosis 
Meningitis  days  1  or  2 
Encephalitis  days  1  or  2 
TIA   (Sx's  <  24  hr)    or  CVA  > 

2  weeks  PTA 
CT  scan  -  Subarachnoid  bleed 
Subdural  hematoma 
Brain  cancer  (primary 
or  metastases) 


X(2,4) 
X 


X 
X 
X 
X 

X 
X 
X 


X(l) 
X 


Renal 

Chronic  dialysis 
Kidney  transplant 


X 
X 


Metabolic 

Thyrotoxicosis 
Thiamine  deficiency 
Paget 's  disease 


X 
X 
X 
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Table  A. 3--continued 


Criterion  CHF  AMI       PNE  CVA  HIP 

Musculoskeletal 

Bilateral  hip  fracture  at  X 
admission 

Femoral  shaft  fracture  alone  X 
Acetab/pelvic  fracture  alone  X 
Prior  fracture  or  surgery  in  X 

currently  fractured  hip 
Known  cancer  or  metastases  in  X 

fracture  hip 

Other 

Cancer  newly  Dx'd  during  the  X 
admission 

Active  cancer  at  admission  X 
Severe  burns  at  admission 

Trauma  is  reason  for  admission  X  XX 

Major  surgery  performed  during  X  X 

the  admission 

(1)  Present  on  any  day  1  EKG. 

(2)  Within  48  hours  before  admission. 

(3)  In  the  emergency  room. 

(4)  Newly  diagnosed  at  admission. 
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(1)  Shortness  of  breath,  dyspnea,  respiratory  distress,  orthopnea,  paroxysmal 
nocturnal  dyspnea  (PND),  tachypnea,  rapid  respiratory  rate,  difficulty 
breathing,  wheezing. 

Current  diagnosis  of: 

(2)  Pulmonary  edema. 

(3)  CHF,  congestive  failure,  left  or  right  heart  failure  (LHF,  RHF)  or  decom- 
pensation, left  or  right  ventricular  failure  (LVF,  RVF)  or  decompensation. 

(4)  Anasarca. 

(B)  (1)  Chest  X-ray  report  on  days  1  or  21  describing  CHF,  congestive  failure,  left 
or  right  heart  failure  (LHF,  RHF)  or  decompensation,  left  or  right  ventric- 
ular failure  (LVF,  RVF)  or  decompensation;  engorgement,  distention, 
plumpness,  overload,  increase,  redistribution,  congestion,  prominence  of 
pulmonary  or  hilar  vessels  (vasculature);  increased  pulmonary  venous 
pressure;  pulmonary  congestion;  interstitial  or  peribronchial  or  perivascu- 
lar edema/congestion/infiltrate;  Kerley  B  lines;  peribronchial  cuffing;  pul- 
monary edema;  edema  in  lung  bases. 

(2)  Current  signs  on  physical  exam  of  anasarca,  generalized  edema,  edema  or 
swelling  in  lower  extremities,  pedal  edema,  ankle  edema  or  swelling,  pre- 
tibial  edema.  The  edema  may  be  described  as  pitting  or  nonpitting  or  X+ 
(where  X  =  any  number  from  1  to  4)  but  is  not  described  as  unilateral. 

Congestive  Heart  Failure  Exclusion  Criteria 

A.  Reasons  for  exclusion  related  to  sampling: 

1.  ICD-9-CM  code  consistent  with  congestive  heart  failure  is  not  listed. 

2.  Patient  <  65  years  old. 

3.  Patient  not  hospitalized  in  specified  study  year. 

4.  Patient  transferred  to  this  hospital  from  another  acute  care  general  hospital's  inpa- 
tient service. 

5.  Congestive  heart  failure  developed  during  the  hospitalization  and  was  not  the  rea- 
son for  admission. 

B.  Clinical  exclusions  because  patients  with  these  conditions  are  significantly  different 
from  other  patients  with  CHF  who  will  constitute  the  sample: 

1.  Thyrotoxicosis. 

2.  Thiamine  deficiency. 

3.  Paget's  disease. 

4.  Chronic  dialysis/renal  transplant. 

C.  Clinical  exclusions  because  the  presence  of  these  conditions  will  be  the  major  deter- 
minants of  processes  and  outcomes  during  this  hospitalization  (rather  than  the  presence 
of  CHF): 

If  no  chest  X-ray  report  from  days  1  or  2,  use  the  physician  description  of  the  admission  chest  X-ray  found  in  the 
history  and  physical,  admitting  note,  consult  reports,  and  discharge  summary. 
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1.  Acute  MI  present  on  admission  as  noted  by  any  EKG  on  day  1  of  admission  showing 
an  acute  MI. 

2.  Acute  CVA  within  48  hours  before  admission. 

Note:    Patients  who  have  a  new  CVA  noted  on  admission  are  excluded. 

3.  Arrest  within  48  hours  before  admission. 

Note:    Patients  who  arrest  in  the  emergency  room  are  not  excluded. 

4.  Coma  present  on  admission. 

5.  Coronary  or  valvular  surgery  during  the  admission. 

6.  Cancer  known  to  be  active  at  admission. 

7.  Multiple  trauma  as  reason  for  admission. 

8.  Major  surgery  occurred  during  the  admission. 

Acute  Myocardial  Infarction  Inclusion  Criteria 

1.    Medical  record  for  this  admission  shows  any  of  the  following  ICD-9-CM  codes  for 
acute  myocardial  infarction: 

410.XX 

X  is  any  value  0  to  9  or  no  value. 
and  2.    Patient  has  an  acute  MI  as  reason  for  admission. 

Acute  Myocardial  Infarction  Exclusion  Criteria 

A.  Reasons  for  exclusion  related  to  sampling: 

1.  ICD-9-CM  code  consistent  with  MI  is  not  listed. 

2.  Patient  <  65  years  old. 

3.  Patient  not  hospitalized  in  specified  study  year. 

4.  Patient  transferred  to  this  hospital  from  another  acute  care  general  hospital's  inpa- 
tient service. 

5.  MI  occurred  during  the  hospitalization  and  was  not  the  reason  for  admission. 

B.  Reasons  for  exclusion  related  to  systemic  illness: 
None. 

Pneumonia  Inclusion  Criteria 

1.    Medical  record  for  this  admission  shows  any  of  the  following  ICD-9-CM  codes  for 
pneumonia: 

480.9X 

481.  XX 

482.  XX 

483.  XX 
485.XX 
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486.XX 
487.0X 
507.0X 
510.XX 
511.1X 
513.0X 
799.  IX 

X  is  any  value  0  to  9  or  no  value. 

and  2.    Patient  has  pneumonia  as  reason  for  admission.  The  presence  of  pneumonia  on 
admission  is  noted  as  at  least  one  from  (A)  and  at  least  one  from  (B): 

(A)  Infiltrate  or  pneumonia  noted  on  the  physician's  note  or  radiologist's  report  of 
the  preadmission  or  admission  chest  X-ray. 

(B)  Any  of  the  following  symptoms  or  signs  noted  on  the  day  of  admission: 

(1)  fever  a  101°  F  or  38.2°  C. 

(2)  WBC  a  15,000. 

(3)  WBC  differential  a  90  percent  polys  (regardless  of  WBC). 

(4)  WBC  differential  a  15  percent  bands  (regardless  of  WBC). 

(5)  New  onset  cough  or  sputum  production. 

Pneumonia  Exclusion  Criteria 

A.  Reasons  for  exclusion  related  to  sampling: 

1.  ICD-9-CM  code  consistent  with  pneumonia  is  not  listed. 

2.  Patient  <  65  years  old. 

3.  Patient  not  hospitalized  in  specified  study  year. 

4.  Patient  transferred  to  this  hospital  from  another  acute  care  general  hospital's  in- 
patient service. 

5.  Pneumonia  developed  during  the  hospitalization  and  was  not  the  reason  for  admis- 
sion. 

B.  Clinical  exclusions  because  patients  with  these  clinical  conditions  are  significantly  dif- 
ferent from  other  patients  with  pneumonia  who  will  constitute  the  sample: 

1.  Primary  or  metastatic  lung  cancer  known  at  admission. 

2.  Interstitial  lung  disease  known  at  admission. 

3.  Active  pulmonary  tuberculosis  known  at  admission. 

C.  Clinical  exclusions  because  the  presence  of  these  conditions  will  be  the  major  deter- 
minants of  processes  and  outcomes  during  this  admission  (rather  than  the  presence  of 
pneumonia): 

1.  Acute  MI  present  on  admission  as  noted  by  any  EKG  on  day  1  of  admission  showing 
an  acute  MI. 

2.  Arrest  within  48  hours  before  admission  because  it  cannot  be  determined  if  the  pri- 
mary event  was  pulmonic  or  cardiac. 
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Note:  Patients  who  arrest  in  the  emergency  room  are  not  excluded  when  they  have 
an  apparent  respiratory  arrest  as  a  sequel  to  the  pneumonia  which  was  the 
reason  for  the  admission. 

3.    Major  surgery  occurred  during  this  admission. 

Note:  Patients  with  surgical  procedures  which  are  performed  as  part  of  the 
management  of  the  pneumonia  (i.e.,  lung  biopsy,  bronchoscopy)  will  not  be 
excluded. 


Cerebrovascular  Accident  Inclusion  Criteria 

1.  Medical  record  for  this  admission  shows  any  of  the  following  ICD-9-CM  codes  for 
cerebrovascular  accident: 

431.XX 
434.XX 
436.XX 

X  is  any  value  0  to  9  or  no  value. 

and  2.  Patient  has  new  CVA  as  reason  for  admission.  The  presence  of  a  new  onset  CVA  is 
defined  as  the  presence  of  new  onset  (a  2  weeks  before  admission)  paresis,  paralysis, 
or  weakness  lasting  more  than  24  hours. 

Cerebrovascular  Accident  Exclusion  Criteria 

A.  Reasons  for  exclusion  related  to  sampling: 

1.  ICD-9-CM  code  consistent  with  CVA  is  not  listed. 

2.  Patient  s  65  years  old. 

3.  Patient  not  hospitalized  in  specified  study  year. 

4.  Patient  transferred  to  this  hospital  from  another  acute  care  general  hospital's  in- 
patient service. 

5.  Acute  CVA  occurred  during  the  hospitalization  and  is  not  the  reason  for  admission. 

6.  Acute  CVA  occurred  more  than  2  weeks  before  admission,  i.e.,  patient  had  CVA 
diagnosed  or  symptoms  of  paralysis,  paresis,  or  weakness  a  2  weeks  before  admis- 
sion. 

7.  Patient  did  not  have  a  stroke,  but  had  a  TIA,  i.e.,  symptoms  of  paresis,  paralysis, 
weakness  were  transient,  lasting  s  24  hours. 

B.  Clinical  exclusion  because  patient  has  a  competing  clinical  condition  which  will  be  the 
predominant  determinant  of  processes  and  outcomes,  rather  than  CVA: 

1.  Multiple  sclerosis. 

2.  Head  trauma. 

3.  Chronic  dialysis. 

4.  Meningitis  known  on  days  1  or  2  of  admission. 

5.  Encephalitis  known  on  days  1  or  2  of  admission. 
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6.    C-T  scan  shows  any  of  the  following: 

a.  subarachnoid  bleed. 

b.  subdural  hematoma. 

c.  primary  or  metastatic  brain  cancer. 

Hip  Fracture  Inclusion  Criteria 

1.    Medical  record  for  this  admission  shows  any  of  the  following  ICD-9-CM  codes  for  hip 
fracture: 

820.0X  (except  820.01) 

820.2X 

820.8X 

X  is  any  value  0  to  9  or  no  value. 
and  2.    Patient  has  a  hip  fracture  as  reason  for  admission. 

Hip  Fracture  Exclusion  Criteria 

A.  Reasons  for  exclusion  related  to  sampling: 

1.  ICD-9-CM  code  consistent  with  hip  fracture  is  not  listed. 

2.  Patient  <  65  years  old. 

3.  Patient  not  hospitalized  in  specified  study  year. 

4.  Patient  transferred  to  this  hospital  from  another  acute  care  general  hospital's  in- 
patient service. 

5.  Hip  fracture  occurred  during  the  hospitalization  and  was  not  the  reason  for  admis- 
sion. 

B.  Clinical  exclusions  because  patients  with  this  hip  pathology  are  significantly  different 
from  other  patients  with  hip  fracture  who  will  be  in  the  sample: 

1.  Known  cancer/metastases  in  fractured  hip. 

2.  Current  bilateral  hip  fracture. 

3.  Femoral  shaft  fracture  without  hip  fracture. 

4.  Acetabular/pelvic  fracture  without  hip  fracture. 

5.  Prior  fracture  or  surgery  in  currently  fractured  hip. 

C.  Clinical  exclusion  because  the  presence  of  these  conditions  will  be  the  major  deter- 
minants of  processes  and  outcomes  during  this  hospitalization  (rather  than  the  presence 
of  the  hip  fracture): 

1.  Acute  MI  within  48  hours  before  admission. 

2.  Arrest  within  48  hours  before  admission. 

Note:    Patients  who  arrest  in  the  emergency  room  are  excluded. 

3.  Acute  CVA  within  48  hours  before  admission. 

4.  Coma  at  admission. 
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5.  Multiple  trauma  as  reason  for  admission. 

6.  New  cancer  (other  than  basal  cell)  diagnosed  during  the  admission. 

FLOW  DIAGRAMS  REPRESENTING  DISEASE-SPECIFIC  EXCLUSION  CRITERIA 

Tables  A.4  through  A.8  are  flow  diagrams  for  each  disease-specific  exclusion  criterion. 

DISTRIBUTION  OF  PATIENTS  BY  CLINICAL  EXCLUSION  CRITERIA 
FOR  PRE-  AND  POST-PPS  SAMPLE  COMBINED 

Patients  were  selected  for  possible  inclusion  in  the  study  sample  if,  according  to  hospital 
records,  the  patient  was  hospitalized  in  one  of  the  study  years.  The  medical  record  of  each  of 
these  patients  was  reviewed  to  verify  that  the  patient  did  meet  the  disease-specific  clinical 
inclusion  and  exclusion  criteria.  Tables  A.9  through  A.  14  show  the  distribution  of  cases 
excluded  for  each  disease  by  reason  for  exclusion  and  study  period. 

Table  A.  15  shows  the  most  frequent  reasons  that  medical  records  were  excluded  from 
the  study  sample  across  diseases.  This  summary  allows  comparison  of  the  frequency  of 
exclusions  across  diseases  for  the  pre-  and  post-PPS  sample  combined. 

ACUTE  CARE  HOSPITAL  TRANSFERS  AND  PPS  EFFECT 

As  noted  in  Table  A.  15,  we  excluded  records  from  the  analysis  if  patients  were 
transferred  to  the  study  hospital  from  another  acute  care  general  hospital,  since  in  these 
cases  we  would  not  have  access  to  the  medical  record  from  the  initial  hospital  stay.  The  table 
shows  that  the  fraction  of  patients  excluded  because  of  transfers  dropped  somewhat  in  the 
post-PPS  period  for  every  disease  except  myocardial  infarction.  This  decline  is  statistically 
significant  for  stroke  and  approaches  significance  both  for  hip  fracture  and  for  all  diseases 
combined;  the  increase  in  transfer  rate  for  AMI  is  statistically  significant  in  the  other  direc- 
tion. This  evidence  suggests  that  PPS  implementation  has  affected  the  rate  of  patient 
transfers  between  acute  care  general  hospitals.  The  declines  in  transfer  rates  are  consistent 
with  financial  incentives  provided  by  PPS. 

For  hip  fracture,  we  also  noted  a  reduction  in  the  number  of  patients  initially  hospital- 
ized for  the  fracture  at  one  of  the  study  hospitals  but  then  transferred  elsewhere  for  surgery. 
During  the  pre-PPS  period  5.4  percent  of  patients  were  transferred  from  one  of  the  study 
hospitals  for  surgery.  This  number  dropped  to  1.7  percent  of  patients  in  the  post-PPS  period. 

Review  of  the  hospital-generated  lists  of  patients  organized  by  ICD-9-CM  code  and  year 
provides  additional  supportive  evidence  for  the  trend.  During  the  pre-PPS  period,  five  of  the 
297  study  hospitals  admitted  no  patients  with  hip  fractures,  whereas  11  hospitals  during  the 
post-PPS  period  had  no  such  patients.  This  suggests  that  before  PPS  implementation,  some 
hospitals  admitted  patients  with  hip  fractures  and  transferred  them  for  surgery  as  needed. 
Following  PPS  implementation,  it  appears  that  hospitals  without  surgical  facilities  were  less 
likely  to  admit  such  patients.  In  the  latter  period,  it  appears  patients  were  triaged  to  hospi- 
tals with  surgical  capabilities  before,  rather  than  during,  admission  to  the  hospital  without 
surgical  capability. 

During  the  pre-PPS  period,  both  the  initial  and  receiving  hospitals  were  reimbursed  in 
proportion  to  their  charges.  However,  under  DRG-based  prospective  payment,  the  fixed 
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Table  A. 9 


REASONS  FOR  EXCLUSION: 

CONGESTIVE  HEART 

FAILURE 

Distribution  of 

Reason  for  Exclusion 

All 
(n=3740) 

Pre-PPS 
(n=1915) 

(n=1825) 

Not  admitted  during 
study  periods  (1) 

0.3 

— 

— 

No  eligible  ICD-9-CM  code 

1.0 

1.5 

0.6 

Age  less  than  65 

0.2 

0.2 

0.3 

acute  care  hospital 

1.8 

2.0 

1.5 

wo  symptoms  ol  ^nr    ai  aumission 

5.2 

7.2 

J  .  u 

Neither  chest  X-ray  evidence 
for  CHF  nor  leg  edema 

11.6 

14  .0 

9.1 

Acute  myocardial  infarction 
at  admission 

0.7 

1 . 1 

0  . 3 

Thyrotoxicosis  or  chronic 
dialysis  or  renal  transplant 
or  arrest  immediately  before 
admission  or  major 
surgery  during  admission 

1.3 

1.1 

1.7 

Poor  prognosis  cancer  or 
active  cancer  or  current 
chemotherapy  or  radiotherapy 

1.8 

1.2 

2.4 

(1)   This  is  the  total  number  of  cases  excluded/total  number  of  cases 
included  and  excluded  regardless  of  study  period  since  we  do  not  have 
an  accurate  year  of  hospitalization  for  these  patients. 
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Table  A. 10 

REASONS  FOR  EXCLUSION:      ACUTE  MYOCARDIAL  INFARCTION 


Distribution  of 

Cases,  % 

Reason  for  Exclusion 

All 
(n=3403) 

Pre-PPS 
(n=1717) 

Post-PPS 
(n=1686) 

Not  admitted  during 
study  periods  (1) 

0.4 



No  eligible  ICD-9-CM  code 

1 .  5 

2.1 

0.8 

Age  less  than  65 

0.3 

0.1 

0.4 

Transferred  from  another 
acute  care  hospital 

4  . 1 

3.3 

4.9 

No  symptoms  of  AMI  at  admission 

6.5 

8.0 

5.0 

No  symptoms  of  AMI  during 

the  24  hours  immediately  before 

admission 

2.2 

2.3 

2.0 

(1)   This  is  the  total  number  of  cases  excluded/total  number  of  cases 
included  and  excluded  regardless  of  study  period,   since  we  do  not  have 
an  accurate  year  of  hospitalization  for  these  patients. 
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Table  A. 11 
REASONS  FOR  EXCLUSION:  PNEUMONIA 


Distribution  of 

Cases,  % 

Reason  for  Exclusion 

All 
(n=4381) 

Pre-PPS 
(n=2184) 

Post-PPS 
(n=2197) 

Not  admitted  during 
study  periods  (1) 

0.2 

— 

No  eligible  ICD-9-CM  code 

1.6 

1.9 

1.2 

Age  less  than  65 

0.5 

0.5 

0.5 

Transferred  from  another 
acute  care  hospital 

1.8 

1.9 

1.7 

No  chest  X-ray  evidence 
for  pneumonia 

22.  5 

23.7 

21.1 

No  symptoms,   signs,  or 
laboratory  data  suggesting 
pneumonia  at  admission 

5.0 

4.9 

5.1 

Acute  myocardial 
infarction  at  admission 

0.2 

0.2 

0.1 

Known  lung  cancer  diagnosis 
or  arrest  immediately 
before  admission  or  major 
surgery  during  admission 

3.6 

3.1 

4.1 

(1)   This  is  the  total  number  of  cases  excluded/total  number  of  cases 
included  and  excluded  regardless  of  study  period,   since  we  do  not  have 
an  accurate  year  of  hospitalization  for  these  patients. 
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Table  A. 12 

REASONS  FOR  EXCLUSION:      CEREBROVASCULAR  ACCIDENT 


Distribution  of  Cases,  % 


All  Pre-PPS  Post-PPS 

Reason  for  Exclusion  (n=3432)        (n=1772)  (n=1660) 


Not  admitted  during 
study  periods  (1) 

0.3 

No  eligible  ICD-9-CM  code 

1.2 

1 

.  4 

1.0 

Age  less  than  65 

0.3 

0 

mm 

0.4 

Transferred  from  another 
acute  care  hospital 

3.3 

4 

.  4 

2.2 

No  symptoms  or  signs  of 
CVA  at  admission 

5.5 

7 

.5 

3.4 

CVA  symptoms  or  signs 
persist  <  24  hours 

2.5 

2 

.2 

2.8 

Onset  of  CVA  symptoms  >  14  days 
before  admission 

3.0 

4 

.1 

1.9 

Subdural  hematoma,   head  trauma, 
meningitis,   encephalitis,   or  brain 
abscess  at  admission;   or  history 
of  central  nervous  system  malignancy, 
chronic  dialysis,   or  transplant 

0.6 

0 

.  6 

0.7 

Acute  myocardial  infarction 
at  admission 

0.3 

0 

.5 

0.2 

(1)   This  is  the  total  number  of  cases 
included  and  excluded  regardless  of  study 

excluded/ total 
period,  since 

number 
we  do 

of  cases 
not  have 

an  accurate  year  of  hospitalization  for  these  patients. 
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Table  A. 13 
REASONS  FOR  EXCLUSION:      HIP  FRACTURE 


Distribution  of 

Cases,  % 

Reason  for  Exclusion 

All 
(n=3343) 

Pre-PPS 
(n=1694) 

Post-PPS 
(n=1649) 

Not  admitted  during 
study  periods  (1) 

0.3 

No  eligible  ICD-9-CM  code 

1.5 

2.2 

0.7 

Age  less  than  65 

0.1 

0.1 

0.1 

Transferred  from  another 
acute  care  hospital 

2.2 

2.6 

1.2 

No  hip  fracture  at  admission 

1.9 

2.0 

2.5 

Bilateral  hip  fractures  or 
multiple  trauma  or  known  hip 
cancer  or  prior  ipsilateral  hip 
fracture 

10.2 

11.1 

9 .  3 

Transferred  to  another  hospital 
for  surgery 

3.6 

5.4 

1.7 

Arrest  or  coma  at  admission 

0.1 

0.1 

0.1 

Acute  myocardial  infarction 
at  admission 

0.2 

0.1 

0.3 

(1)   This  is  the  total  number  of  cases  excluded/total  number  of  cases 
included  and  excluded  regardless  of  study  period,   since  we  do  not  have 
an  accurate  year  of  hospitalization  for  these  patients. 
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Table  A. 14 
REASONS  FOR  EXCLUSION:  DEPRESSION 


Distribution  of 

Cases,  % 

Reason  for  Exclusion 

All 
(n=3626) 

Pre-PPS 
(n=2045) 

Post-PPS 
(n=1581) 

Not  all  volumes  of  medical 

record  for  study  are  available  (1) 

0.5 

0.0 

0 . 1 

Not  admitted  during  study  periods  (2) 

0.2 

No  eligible  ICD-9-CM  code 

2  .  5 

2  .  6 

2.2 

Age  less  than  65 

0.7 

0  .  5 

0 . 9 

No  depression  at  admission 

10.9 

12.7 

8.6 

Arrest,   acute  myocardial  infarction, 
cerebrovascular  accident,   or  major 
trauma  at  admission;   or  chronic 
dialysis;    or  major  surgery 
during  admission 

2.6 

2.2 

3.0 

Poor  prognosis  or  active  cancer, 
or  current  chemotherapy  or 
radiotherapy 

1.2 

1.0 

1.6 

(1)   The  pre   (0  percent)   and  post   (0.1  percent)   do  not  average  to  the 
all  patients'  value   (0.5  percent)   because  18  of  the  19  records  that 

were  excluded  with  "not  all  volumes  of  the  medical  record  for  the  study 
hospitalization  were  available"  were  not  eligible  for  one  of  the  study 
years . 


(2)   This  is  the  total  number  of  cases  excluded/total  number  of  cases 
included  and  excluded  regardless  of  study  period,   since  we  do  not  have 
an  accurate  year  of  hospitalization  for  these  patients. 
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payment  is  divided  between  the  two  hospitals,  providing  a  disincentive  to  receiving  hospitals 
for  transfer  of  expensive  patients. 

ACTUAL  HOSPITAL  AND  PATIENT  SAMPLE  PLAN 

Because  the  actual  sample  could  not  be  finalized  until  after  patient  records  were 
reviewed  for  clinical  inclusion  and  exclusion  criteria,  the  details  of  the  sampling  plan  at  the 
patient  level  within  hospitals  were  designed  to  adapt  to  the  number  of  included,  excluded, 
and  not  available  records  found  in  the  field.  In  each  hospital  we  set  targets  for  the  number  of 
records  planned  by  disease  and  study  year,  as  exemplified  in  Table  A.  16.  Each  hospital  had  a 
3  by  6  table  of  sampling  goals  similar  to  the  ones  in  Table  A.  16.  Records  were  abstracted  in 
each  cell  of  a  given  hospital's  table  until  the  goal  was  met  or  the  available  supply  of  relevant 
records  was  exhausted.  In  the  latter  case,  abstractors  were  instructed  to  choose  another 
record  with  the  same  disease  and,  when  possible,  in  the  same  pre-  or  post-PPS  time  period 
(e.g.,  if  the  target  for  pneumonia  in  1982  could  not  be  met,  the  shortfall  was  made  up  if  possi- 
ble with  pneumonia  records  from  1981).  In  73  percent  of  the  hospital-disease  combinations, 
the  initial  target  set  by  RAND  was  met  without  adjustments  (see  Table  A.  17). 

SPECIAL  FEATURES  OF  THE  SAMPLING  PLAN 

The  sampling  plan  we  employed  had  four  features  requiring  special  attention  during 
the  analysis:  our  oversampling  of  poor  patients,  the  clustering  of  our  patients  in  a  relatively 
small  number  of  hospitals,  the  longitudinal  nature  of  the  design  at  the  hospital  level,  and  the 
aggregation  of  our  results  across  diseases.  We  now  describe  these  features  and  how  we  dealt 
with  them.  This  discussion  extends  the  one  in  Sec.  II  and  provides  additional  details. 

OVERSAMPLING  OF  POOR  PATIENTS 

Our  oversampling  of  patients  from  hospitals  serving  an  unusually  large  fraction  of 
Medicaid  patients  would  yield  somewhat  biased  raw  findings  if  the  quality  of  care  in  high- 
Medicaid  facilities  differs  substantially  from  that  in  other  hospitals.  To  arrive  at  nationally 
representative  findings,  it  was  necessary  to  reweight  our  raw  patient-level  results,  giving 
less  weight  to  patients  from  high-Medicaid  facilities.  We  describe  in  Sec.  II  how  we  calcu- 
lated the  weights  for  this  purpose.  The  weighted  and  unweighted  results  differed  little  in 
most  of  our  major  analyses;  we  therefore  presented  unweighted  findings  (unless  otherwise 
indicated). 

The  only  major  result  that  was  even  partially  affected  by  this  reweighting  arose  in  the 
analysis  of  outcomes  (Sec.  VII).  Unweighted  mortality  rates  30  days  after  admission, 
adjusted  for  sickness  at  admission,  for  our  study  diseases  combined  (excluding  depression) 
were  16.7  percent  pre-PPS  and  15.7  percent  post-PPS.  This  difference  of  1.0  percentage 
points  (95  percent  confidence  interval  -0.1  percent  to  2.1  percent)  was  significant  at  the  p  = 
0.07  level.  When  we  reweighted  to  the  nation,  downweighting  the  high-Medicaid  hospitals, 
the  pre-  and  post-PPS  30-day  adjusted  mortality  rates  aggregating  across  our  five  diseases 
became  16.5  percent  and  15.4  percent,  respectively.  The  reweighted  difference  was  thus  1.1 
percentage  points  (95  percent  confidence  interval  0.0  percent  to  2.2  percent),  and  the  p-value 
dropped  to  0.04.  This  discrepancy  will  give  pause  only  to  readers  who  believe  that  there  is 
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Table  A. 16 


PATIENT   SAMPLING  TARGETS  AT  THE  HOSPITAL  LEVEL 
(a)   The  idealized  table 
HIP       AMI       CHF       PNE       CVA       DEP  total 

1981  |    1.9   I    1.9   |    1.9   |    1.9   |    1.9   |    1.9   I  11.3 

1982  |  2.8  |  2.8  I  2.8  I  2.8  I  2.8  I  2.8  I  17.0 
1985/6    |    4.7    |    4.7    |    4.7    |    4.7    |    4.7    [    4.7    |  28.3 

total       9.4       9.4       9.4       9.4       9.4       9.4  56.7 
(b)   A  typical  table  for  a  nondepression  hospital 
HIP       AMI       CHF       PNE       CVA       DEP  total 

1981  |     2     |     2     |     2     |.     2     |     2     |     2     |  12 

1982  |  2  |  3  |  2  |  3  |  3  |  4  |  17 
1985/6   |     6     I     5     |     6     |     5     I     5     |     2     |  29 

total       10         10         10         10         10  8  58 

(c)   A  typical  table  for  a  depression  hospital 

HIP       AMI        CHF       PNE       CVA       DEP  total 

1981  |     2     |     1     |     1     |     1     |     2     |     5     |  12 

1982  |  2  |  2  |  2  |.  2  |  2  |  7  |  17 
1985/6   |     4     |      4     |      4      |     4      |     4      |    10      |  30 

total         8  7  7  7  8         22  59 


NOTE:     These  targets  were  based  on  a  20  percent,   30  percent,   and  50 
percent  allocation  of  records  in  the  study  years  1981,   1982,   and  1985/6 
(with  a  roughly  equal  division  of  the  50  percent  post-PPS  records  between 
1985  and  1986) ,   and  a  goal  of  balance  across  diseases  conditional  on  the 
depression  status  of  the  hospital.     The  number  of  records  targeted  at  a 
given  hospital  varied  from  57  to  60,   depending  on  the  hospital's  size, 
to  allow  for  a  modest  shortfall  of  the  sampling  goals  in  some  of  the 
very  smallest  hospitals. 
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Table  A. 17 

HOSPITAL-LEVEL  COMPARISON  OF   SAMPLING  GOALS 
AND  ACTUAL  SAMPLE  CHOSEN 


Hospitals 

Disease  Result  (n=297)  Number  of  Records 


Acute 

Target 

met--no  adjustments 

253 

1 1 1  y  w  \_*  ex  -i_  v_-i  _i_  cx  _i_ 

Ta  rge t 

m^t"    wi  1~  hi    Pi  c\  "i  n  s  t"  m^n  t~  s 

-  L  I  - —  I            W  _!_  L  ii        C-l           J  U  O  I—  I  I  L'      i!  L 

27 

infarct"  ion 

Ta  rge t 

not"  met" 

13 

-24 

( 0 

8% 

of 

1"  >R  rfY^t"  ^ 
l.  o  j_  y  ^  l-  / 

Target 

surDa  s  sed 

4 

+  4 

Congestive 

Target 

met --no  adjustments 

223 

hea  irt 

Ta  rget 

m^l-    wi  t"  h    a  H  n  n  <?t" rnpn t*  s 

X  L  L  V — .              VT  -L.  1      II        CA  VJ.     I  I— I  t_>  l_  1  I         1!  L  kJ 

49 

fa  i  In  r~£* 

J_  Cj.  —1  1_  U.  J-  ^ 

Target 

not     mP> 1" 

18 

2% 

of 

\J  1_ 

f-  a  TTTO+"  ^ 

Target 

snrna  s  seci 

lw*          -k—  k— '  V-A  V_>L 

7 

+  8 

Hip 

Target 

met — no  adjustments 

244 

fracture 

Ta  rget 

met  with  adiustments 

29 

Ta  rge t 

not  met 

24 

-45 

(1 

6% 

of 

h a.  rrret ) 

Target 

surpassed 

0 

o 

Pneumonia 

Target 

met — no  adjustments 

157 

Target 

met  with  adjustments 

85 

Target 

not  met 

45 

-112 

(3. 

9% 

of 

target ) 

Target 

surpassed 

10 

+  10 

Cerebro- 

Target 

met--no  adjustments 

243 

vascular 

Target 

met  with  adjustments 

41 

accident 

Target 

not  met 

7 

-19 

(0. 

7% 

of 

target ) 

Target 

surpassed 

6 

+  7 

Depression 

Target 

met--no  adjustments 

181 

Target 

met  with  adjustments 

49 

Target 

not  met 

62 

-107 

(3. 

7% 

of 

target ) 

Target 

surpassed 

5 

+  9 
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something  special  about  the  0.05  level.  In  fact,  the  weight  of  evidence  about  the  difference  in 
30-day  adjusted  mortality  rates  for  our  five  diseases  associated  with  the  introduction  of  PPS 
is  essentially  the  same  with  and  without  weighting. 

Hospital-Level  Clustering 

The  clustering  of  our  sampled  patients  in  only  five  states,  30  areas,  and  297  hospitals 
has  implications  for  the  accuracy  of  our  results.  If  there  is  more  similarity  on  average 
between  two  patients  in  the  same  state,  area,  or  hospital  in  the  care  they  receive  than 
between  two  patients  in  different  states,  areas,  or  hospitals,  and  no  adjustment  for  this  intra- 
cluster  correlation  is  made,  the  result  will  be  an  overstatement  of  the  precision  of  our 
findings.  To  adjust  for  the  clustering  and  to  produce  standard  errors  that  accurately  reflect 
the  information  content  of  our  sample,  we  (a)  calculated  provisional  standard  errors  for  all  of 
our  major  estimates  as  if  we  had  gathered  our  data  using  simple  random  sampling,  (b)  com- 
puted "inflation  factors"  based  on  intracluster  correlations  that  measured  the  amount  of 
information  in  our  sample  relative  to  that  obtained  in  a  simple  random  sample  of  the  same 
size,  and  (c)  multiplied  our  provisional  standard  errors  by  the  inflation  factors,  thereby 
adjusting  the  significance  of  our  results  downward. 

To  compute  the  inflation  factors  in  step  (b)  of  this  analysis,  we  performed  analyses  of 
covariance  with  the  following  factors:  state;  area  nested  within  state;  hospital  nested  within 
area,  pre-  and  post-PPS  crossed  with  state,  area,  and  hospital;  patient  nested  within  the 
cross  of  pre-  and  post-PPS  and  hospital;  and  any  patient-specific  covariates  needed  for 
adjustment  purposes  such  as  sickness  at  admission.  State  and  area  were  typically  not  as 
important  as  hospital  in  determining  the  magnitude  of  the  information  loss  resulting  from 
clustering,  so  the  inflation  factors  were  given  approximately  by  the  square  root  of  the  expres- 
sion [1  +  (M  -  1)  r],  where  M  is  the  average  number  of  patients  per  hospital  and  r  is  the 
intracluster  correlation  for  the  given  outcome. 

The  inflation  factors  ranged  from  1.1,  for  outcomes  such  as  in-hospital  and  30-day 
death,  to  2.9  for  our  overall  explicit  process  score,  aggregating  across  diseases.  The  latter 
inflation  factor,  which  derives  from  an  unusually  high  intracluster  correlation  of  0.16,  indi- 
cates a  remarkable  degree  of  homogeneity  within  hospitals  in  the  processes  of  care  rendered. 
A  correlation  of  0.16  may  not  seem  large,  but  it  must  be  remembered  that  this  is  the  correla- 
tion between  process  scores  for  two  completely  different  individuals,  not  even  necessarily 
with  the  same  disease,  in  the  same  hospital  treated  in  the  same  (pre-  compared  with  post- 
PPS)  time  period.  The  corresponding  intracluster  correlation  for  30-day  mortality  was  only 
0.003. 

Adjustment  for  the  Longitudinal  Design  at  the  Hospital  Level 

Our  design  was  longitudinal  at  the  hospital  level;  i.e.,  we  gathered  data  in  all  our  hospi- 
tals in  both  the  pre-  and  post-PPS  periods.  This  gives  the  analysis  the  character  of  a  paired 
comparison,  in  which  hospital  is  held  constant.  This  hospital-level  pairing,  like  the  hospital- 
level  clustering,  has  implications  for  the  accuracy  of  our  results.  The  pairing  acted  upon  the 
precision  of  our  findings  in  a  manner  opposite  to  that  of  the  clustering:  The  clustering 
decreased  accuracy,  when  compared  with  independent  simple  random  sampling  in  each  of 
the  pre-  and  post-PPS  periods,  whereas  the  pairing  increased  accuracy,  because  of  correla- 
tions over  time  in  the  patterns  of  care  given  to  patients  within  hospitals. 
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To  account  for  the  pairing,  we  again  fit  analysis  of  variance  models  similar  to  those 
mentioned  above  in  the  clustering  discussion,  to  quantify  the  extent  to  which  data  from  a 
given  hospital  pre-PPS  are  correlated  with  data  from  the  same  hospital  post-PPS.  The 
results  paralleled  those  described  in  the  clustering  discussion:  low  across-period  correlations 
for  death  outcomes  and  high  correlations  for  process. 

As  an  example  of  the  process  results,  the  correlation  between  the  mean  process  score  in 
a  given  hospital  in  the  pre-PPS  period,  aggregating  across  all  five  diseases,  and  the  mean 
process  score  in  the  same  hospital  in  the  post-PPS  period  is  +0.73.  This  is  an  ecological  corre- 
lation, in  which  the  means  being  correlated  are  each  based  on  an  average  of  23.6  patients  per 
hospital;  since  averaging  is  a  smoothing  process  that  damps  out  patient-level  variation,  this 
correlation  is  much  higher  than  the  corresponding  patient-level  process  correlation.  When 
this  effect  was  accounted  for,  we  discovered  that  the  correlation  between  the  process  scores 
for  two  patients  chosen  from  the  same  hospital,  one  pre-PPS  and  one  post-PPS  (the  correla- 
tion relevant  to  the  pairing),  is  quite  close  to  the  correlation  between  the  process  scores  for 
two  patients  chosen  from  the  same  hospital  in  the  same  time  period  (the  correlation  relevant 
to  the  clustering).  This  implied  that  when  we  computed  "deflation  factors,"  analogous  to  the 
clustering  "inflation  factors,"  to  adjust  simple  random  sampling  standard  errors  for  the  pair- 
ing, the  clustering  and  pairing  effects  approximately  canceled  each  other.  Thus,  interest- 
ingly, in  the  end  we  were  able  to  analyze  our  data  essentially  as  if  they  had  been  gathered 
with  simple  random  sampling  in  an  unpaired  fashion. 

Aggregating  Across  Diseases 

Two  issues  arise  in  producing  across-disease  summaries:  reweighting  the  diseases  back 
to  their  actual  frequencies  in  the  Medicare  population,  and  properly  accounting  for  the 
degree  to  which  pre-  and  post-PPS  differences  themselves  differ  by  disease  (i.e.,  accounting 
for  interactions  between  the  pre-  and  post-PPS  and  disease  factors). 

Reweighting  by  disease  prevalence  is  potentially  necessary  because  we  took  samples  of 
roughly  equal  size  in  each  of  our  six  diseases,  even  though  congestive  heart  failure  is  more  than 
three  times  more  frequent  among  Medicare  patients  than  hip  fracture,  for  example.  We  com- 
puted weights  necessary  to  reweight  our  raw  findings  back  to  the  population  of  all  Medicare 
patients  with  one  of  our  study  diseases,  and  did  sensitivity  analyses  to  see  how  much  this 
reweighting  affected  our  results.  In  all  cases  we  examined  that  involved  our  major  results,  the 
weighted  and  unweighted  results  differed  little.  We  therefore  reported  unweighted  findings 
unless  otherwise  noted. 

To  handle  pre-  and  post-PPS  periods  by  disease  interactions,  we  fit  analysis  of  variance 
models  in  which  disease  was  an  explicit  factor  and  interacted  disease  with  the  pre-  and  post- 
PPS  effect.  When  the  interactions  were  large,  i.e.,  when  there  was  serious  disagreement  among 
our  diseases  in  the  size  of  the  difference  between  periods,  we  emphasized  disease-specific  sum- 
maries in  our  written  findings,  instead  of  drawing  the  reader's  attention  to  across-disease  PPS 
effects.  When  the  differences  by  disease  interactions  were  small  to  moderate,  we  conservatively 
reported  the  significance  of  the  pre-  and  post-PPS  effect  by  adjusting  its  significance  in  a 
manner  consistent  with  the  size  of  the  interaction  (Scheffe,  1959).  We  did  this  by  computing  the 
t-statistics  for  the  pre-  and  post-PPS  difference  separately  for  each  disease,  averaging  these  t- 
statistics,  and  adjusting  the  average  t-value  by  a  multiplier  that  depended  on  the  standard 
deviation,  s,  of  the  t-statistics.  This  multiplier  went  down  as  s  went  up,  so  that  our  combined  t- 
statistic  became  more  conservative  (closer  to  0)  as  the  between-disease  variation  in  the  PPS 
effect  increased. 
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CALCULATIONS  OF  STANDARD  ERRORS  FOR  NATIONALLY 
REWEIGHTED  ESTIMATES 

We  computed  the  standard  errors  for  the  nationally  reweighted  estimates  in  a  standard 
way  using  national  prevalences  in  the  stratification  grid  described  in  Sec.  II.  The  weights  are 
fixed  rather  than  random,  and  one  may  simply  use  weighted  least  squares  in  the  usual  way, 
as  follows. 

Instead  of  estimating  p  in  the  model  Y=|3  +  e  by 

$  =  {XTXTlXTY  , 
we  constructed  a  weight  matrix  W  and  used 

P„  =  (XTWXr1XTWY  . 
This  vector  of  estimates  has  covariance  matrix 


1a  =o2(xTwxr1  , 


from  which  standard  errors  of  any  linear  functions  cT$w  of  the  parameter  estimates  (such  as 
the  difference  between  pre-PPS  and  post-PPS  30-day  mortality  rates,  adjusted  for  sickness  at 
admission)  were  routinely  computable  from 

V(cT^w)  =  cTo^c  . 

(See  any  standard  text  on  regression,  e.g.,  Weisberg,  1985;  or  Draper  and  Smith,  1981).  The 
particular  implementation  of  this  idea  we  used  was  the  one  in  the  interactive  statistics  pack- 
age issued  by  Computing  Resource  Center,  1989. 


DEMOGRAPHIC  CHARACTERISTICS  OF  STUDY  PATIENTS  FOR  PRE-  AND 
POST-PPS  SAMPLE  COMBINED 

Table  A.  18  shows  the  demographic  characteristics  of  patients  for  five  study  diseases  for 
the  pre-  and  post-PPS  sample  combined.  Table  A.  19  compares  demographic  characteristics 
for  patients  in  the  pre-PPS  versus  post-PPS  periods. 
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Appendix  B 


DEVELOPING  AND  COLLECTING 
SICKNESS-AT-ADMISSION  VARIABLES 


This  appendix  provides  additional  documentation  about  the  methods  used  for  measur- 
ing sickness  at  admission  (complexity),  presents  data  about  the  distribution  of  patients  in  the 
sample  by  sickness-at-admission  variables  for  the  pre-  and  post-PPS  sample  combined,  and 
presents  data  about  the  distribution  of  patients  in  the  sample  by  sickness-at-admission  vari- 
ables for  the  pre-PPS  versus  post-PPS  samples. 

To  use  mortality  and  other  clinical  outcomes  as  a  meaningful  proxy  for  hospital  quality, 
we  must  know  the  patient's  sickness  at  admission  (complexity)  before  the  delivery  of  inhospi- 
tal  care.  Outcomes  can  then  be  adjusted  for  sickness  at  admission.  We  used  literature 
reviews,  clinical  judgment,  and  consensus  panels  to  identify  variables  that  have  been  found 
in  clinical  studies  to  predict  outcomes.  We  pilot-tested  the  variables  to  see  whether  they 
were  likely  to  be  reported  consistently  during  both  the  pre-  and  post-PPS  period.  We  finally 
abstracted  patient  characteristics  that  were  likely  to  predict  outcomes,  were  sufficiently  fre- 
quent to  be  worth  collecting  from  the  medical  record,  and  were  likely  to  be  recorded  con- 
sistently during  both  the  pre-  and  post-PPS  period. 

To  use  the  multiple  dimensions  of  clinical  care  to  study  sickness  at  admission,  the  data 
base  must  be  an  accurate  and  valid  representation  of  patient  characteristics  at  admission. 
There  are  three  main  ways  to  collect  this  kind  of  data:  from  routine  billing  information, 
through  prospective  data  collection  at  the  time  of  admission,  and  from  the  medical  record. 
From  billing  data  age,  sex,  and  diagnostic  categories  are  available,  but  this  information  is 
usually  not  sufficient  to  provide  an  adequate  adjustment  for  the  effect  of  complexity  on  out- 
come. Prospective  data  collection  can  make  use  of  a  patient  interview  or  a  checklist  com- 
pleted by  the  physician  and  put  into  the  medical  record.  Unfortunately,  such  techniques  can- 
not be  used  to  describe  past  hospital  performance,  and  they  are  not  currently  feasible  for  use 
in  all  hospitals  (Kahn  et  al.,  1988a). 

We  used  medical  record  review  because  it  is  currently  the  most  feasible  system  for  col- 
lecting sickness-at-admission  data  across  time  periods  and  hospitals.  We  were  guided  in  our 
collection  of  these  data  from  the  medical  record  by  seven  basic  principles. 

First,  for  most  variables  describing  sickness  at  admission,  multiple  data  sources  within 
the  medical  record  needed  to  be  reviewed  (i.e.,  physicians'  notes,  nurses'  notes,  X-ray  reports, 
laboratory  results,  electrocardiograms,  and  pathology  findings).  For  example,  to  recognize 
whether  the  patient  is  immunocompromised,  the  patient's  history  was  examined  for  use  of 
the  word  immunocompromise,  for  diagnostic  equivalents  (e.g.,  metastatic  cancer),  and  for 
immunocompromising  drugs  (e.g.,  chemotherapy  or  steroids).  Review  of  the  physical  exami- 
nation provided  evidence  about  cachexia,  debilitation,  or  thenar  wasting,  and  review  of  labo- 
ratory reports  provided  information  about  white  blood  counts  and  albumin  values.  For  each 
sickness-at-admission  variable,  a  precise  clinical  definition  was  developed. 

Second,  we  took  great  efforts  to  avoid  both  overstating  the  admission  level  of  patients 
with  multiple  descriptions  of  illness  and  understating  the  admission  sickness  level  of 
patients  with  only  a  single  description.  For  example,  some  records  described  patients  as  sick 
according  to  a  number  of  factors  (e.g.,  history,  diagnoses,  use  of  medications,  physical 
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examination,  and/or  laboratory  data);  other  records  used  only  one  term  to  describe  an  equally 
sick  patient.  We  developed  reasonable  criteria  for  data  collection  to  allow  us  to  identify 
patients  with  similar  sickness  levels,  despite  more  or  less  information  in  the  record. 

Third,  we  paid  careful  attention  to  medical  dialects  and  recording  practices  used  in  dif- 
ferent segments  of  the  study  sample.  Since  the  study  used  data  from  different  time  periods, 
we  expected  differences  in  recording,  because  incentives  for  recording  have  changed  over 
time.  Different  settings,  such  as  teaching  compared  with  nonteaching  hospitals,  were  also 
considered  because  they  use  different  vocabularies.  We  used  a  pilot  study  of  medical  records 
to  test  variables  for  consistent  patterns  of  recording  before  and  after  PPS  implementation. 

Fourth,  we  used  special  care  when  employing  reports  of  data  rather  than  the  raw  data 
itself.  This  was  especially  important  when  using  reports  of  radiographic  studies.  Review  of 
the  medical  record  allows  access  only  to  the  radiologists'  (and/or  clinicians')  reports  rather 
than  the  X-rays  themselves.  Although  this  problem  is  more  relevant  to  recorded  than  to 
direct  patient  data,  there  is  a  less  standardized  format  for  reporting  disease  criteria  in  the 
X-ray  reports  than  elsewhere.  Because  X-rays  may  provide  valuable  information  about 
patient  sickness  at  admission  (e.g.,  presence  of  congestive  heart  failure  or  pulmonary  edema 
for  patients  hospitalized  with  acute  myocardial  infarction,  pneumonia,  or  cerebrovascular 
accident;  or  presence  of  infarct,  hemorrhage,  or  increased  intracranial  pressure  for  patients 
hospitalized  with  cerebrovascular  accident),  it  was  important  to  develop  a  valid  system  for 
grading  severity  of  X-ray  findings. 

Fifth,  an  approach  was  developed  for  interpreting  unrecorded  or  missing  data.  For 
example,  in  the  admission  history  and  physical,  physicians  customarily  report  all  important 
positive  historical  and  physical  findings,  but  only  pertinent  negative  findings.  This  meant 
that  abstraction  from  the  medical  record  of  many  clinical  characteristics  present  at  the  time 
of  admission  could  identify  the  presence  of  a  clinical  problem  but  could  not  distinguish  the 
absence  of  the  condition  from  the  situation  where  no  data  were  found  in  the  record  about  the 
presence  or  absence  of  the  problem.  Under  these  circumstances  a  bias  could  result  if 
sickness-at-admission  variables  are  not  carefully  selected,  because  a  hospital  with  detailed 
recording  practices  might  appear  to  have  sicker  patients  than  a  hospital  with  less 
comprehensive  ones. 

To  reduce  the  likelihood  of  bias,  we  attempted  to  include  only  measures  of  sickness  at 
admission  from  the  medical  record  that  generally  would  be  recorded  if  the  patient  had  the 
pathology  in  question.  Following  this  rule,  in  most  instances,  it  would  be  possible  to  assign 
normal  values  or  no  comorbidity  if  specific  data  were  absent.  For  example,  if  there  is  no 
report  about  dementia,  the  patient  can  be  considered  to  have  normal  mental  status.  We 
tested  these  decisions  during  the  analysis  phase  by  comparing  whether  patients  with  missing 
data  were  more  or  less  likely  to  die  or  suffer  some  other  adverse  outcome  than  are  patients 
without  missing  data.  After  carefully  selecting  variables  that  would  not  likely  be  subject  to  a 
recording  bias,  we  assumed  that  no  mention  of  a  variable  relating  to  acute  and  chronic  condi- 
tions and  functional  capabilities  was  the  same  as  no  pathology  or  normal  function.  When 
patients  with  missing  data  had  higher  mortality  rates  than  patients  with  data,  we  made 
adjustments  during  the  study  analysis. 

Sixth,  just  as  a  choice  needs  to  be  made  in  studies  such  as  ours  between  disease-specific 
compared  with  nonspecific  sampling,  a  choice  needs  to  be  made  regarding  the  use  of  a  generic 
measure  of  sickness  at  admission  or  a  disease-specific  measure.  Although  APACHE  II  APS, 
which  uses  the  same  items  and  the  same  weights  to  determine  sickness  levels  across 
diseases,  has  been  shown  to  account  for  a  noticeable  amount  of  the  variability  in  hospital  out- 
comes, clinical  principles  suggest  that  such  a  generic  measure  should  be  supplemented  with 
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various  disease-specific  items  and  disease-specific  weights.  Thus,  we  developed  disease- 
specific  measures  to  supplement  APACHE  II  APS  variables. 

Seventh,  it  is  critical  in  measuring  sickness  levels  to  distinguish  sickness  with  which 
patients  present  at  admission  from  sickness  affected  by  inhospital  processes  and  outcomes. 
To  accomplish  this,  we  used  only  data  from  the  initial  assessment  as  sickness-at-admission 
variables.  For  vital  signs,  we  collected  both  the  first  and  worst  value  for  the  day  of  admission 
but  in  analysis  used  only  first  values.  We  used  the  worst  laboratory  values  on  day  1,  since 
the  medical  record  does  not  reliably  document  timing  of  laboratory  studies.  Most  patients, 
however,  had  only  one  of  each  type  of  laboratory  test  on  each  day.  If  there  were  no  labora- 
tory values  for  a  particular  variable  on  day  1,  values  from  day  2  of  the  hospital  stay  were 
acceptable. 

For  this  PPS  study,  in  which  data  were  collected  retrospectively  from  the  medical 
record,  we  altered  some  of  the  definitions  of  the  variables  used  by  George  Washington 
University  (developers  of  the  APACHE  II  system).  Table  B.l  documents  the  differences 
between  RAND's  variable  definitions  and  those  of  George  Washington  University.1 

Table  B.2  shows  the  distribution  of  APACHE  II  APS  variables  for  five  diseases  in  the 
pre-  and  post-PPS  sample  combined.  The  codes  refer  to  variable  categories  used  in  the  PPS 
data  set.  The  scores  refer  to  the  number  of  points  assigned  for  patient  values  in  each 
category.  The  scores  presented  were  developed  by  Knaus  et  al.  (1985). 

Table  R.3  shows  the  distribution  of  patients  by  RAND  Acute  Complexity  (sickness-at- 
admission)  variables  for  the  pre-  and  post-PPS  sample  combined.  These  disease-specific  lists 
of  variables  are  designed  to  measure  acute  morbid  and  comorbid  conditions  present  at  the 
time  of  hospital  admission.  Abbreviation  used  in  Table  B.3  are  listed  below: 


AF 

Atrial  fibrillation 

CXR.CHF 

Chest  X-ray  description  of 

congestive  heart  failure 

CXR.PNE 

Chest  X-ray  description  of  pneumonia 

CPK.HNV 

Highest  normal  value  for  CPK 

at  the  patient's  laboratory 

D1H 

Day  1  of  hospitalization 

D2H 

Day  2  of  hospitalization 

HNV 

Highest  normal  value  for  CPK 

at  the  patient's  laboratory 

PAC 

Premature  atrial  contraction 

Pre  op 

Pre-operative 

V-tach 

Ventricular  tachycardia 

Table  B.4  shows  the  distribution  of  patients  by  RAND  Chronic  Complexity  (sickness-at- 
admission)  variables  for  the  pre-  and  post-PPS  sample  combined.  The  disease-specific  lists  of 
variables  are  designed  to  measure  chronic  morbid  and  comorbid  conditions  present  at  the 
time  of  hospital  admission.  Abbreviations  used  in  Table  B.4  are  listed  below: 

CABS  Coronary  artery  bypass  surgery 

Chemo  Chemotherapy 

Dx  Diagnosis 

Dx-hip  cancer  Diagnosis  of  hip  cancer 


Personal  communication  with  researchers  from  George  Washington,  1987. 
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ESR  Erythrocyte  sedimentation  rate 

Fib  Fibrillation 

02  Oxygen  therapy 

PCTA  Percutaneous  Transluminal  Angioplasty 

Rx  Therapy 


Table  B.5  shows  the  distribution  of  patients  by  Body  System  variables  for  the  pre-  and 
post-PPS  sample  combined.  The  disease-specific  definitions  of  each  variable  are  listed  in  the 
table. 

Table  B.6  shows  the  distribution  of  patients  by  the  number  of  body  systems  with  sick- 
ness at  admission  for  the  pre-  and  post-PPS  sample  combined.  Body  system  sickness  at 
admission  is  defined  in  Table  B.5.  Both  percentages  and  cumulative  (cum)  percentages  are 
listed. 

Table  B.7  shows  the  distribution  of  patients  by  the  variables  used  in  the  Norris 
Coronary  Prognostic  Index  (Norris  et  al.,  1969).  The  codes  refer  to  variable  categories  used 
in  the  PPS  data  set.  The  scores  refer  to  the  number  of  points  assigned  for  patient  values  in 
each  category.  The  scores  presented  were  developed  by  Norris.  Electrocardiograms  from  day 
1  of  the  hospitalization  were  used  to  assign  location  and  extent  of  the  infarct.  If  no  day  1 
electrocardiogram  was  available,  the  day  2  electrocardiograms  were  used.  Abbreviations 
used  in  Table  B.7  are  listed  below: 


EKG:  anterior 

EKG:  inferior 

EKG:  subendocardial 

EKG:  transmural  but 
location  unspecified 

MD:  anterior 


MD:  anterior 
and  inferior 

MD:  inferior 


MD:  subendocardial 


MD:  transmural  but 
location  unspecified 

MI  location/extent 


The  EKG  report  suggests  an  anterior 

myocardial  infarction 
The  EKG  report  suggests  an  inferior 

myocardial  infarction 
The  EKG  report  suggests  a  subendocardial 

myocardial  infarction 
The  EKG  report  suggests  a  transmural 

myocardial  infarction  with 

location  unspecified 
Physician  reads  the  EKG  report  as  showing 

evidence  for  an  anterior  myocardial 

infarction 

Physician  reads  the  EKG  report  as  showing 

evidence  for  an  acute  and  inferior 

myocardial  infarction 
Physician  reads  the  EKG  report  as  showing 

evidence  for  an  inferior  myocardial 

infarction 
Physician  reads  the  EKG  report  as 

suggesting  a  subendocardial  myocardial 

infarction 

Physician  reads  the  EKG  report  as  showing 
evidence  for  a  transmural  myocardial 
infarction  with  location  unspecified 

Location  and  extent  of  myocardial  infarction 
as  noted  by  physician  review  of 
the  electrocardiogram 
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Table  B.6 

NUMBER  OF  BODY  SYSTEM  PROBLEMS  NOTED 
AT  ADMISSION  ACROSS  DISEASES 
(in  percent;  pre-  and  post-PPS  sample  combined) 


CHF  AMI  PNE  CVA  HIP 

(n=2824)  (n=2853)  (n=2749)  (n=2824)  (n=2762) 

Number  of           

Body  System                   %  %  %  %  % 

Problems                %     Cum  %     Cum  %     Cum  %     Cum  %  Cum 


1 

15 

15 

28 

28 

9 

9 

7 

7 

19 

19 

2 

29 

44 

37 

65 

16 

25 

27 

36 

27 

45 

3 

28 

72 

21 

87 

22 

47 

30 

66 

26 

71 

4 

17 

89 

10 

96 

20 

67 

21 

87 

19 

90 

5 

8 

97 

3 

99 

15 

83 

9 

96 

7 

97 

6 

2 

99 

1 

100 

11 

94 

4 

99 

3 

100 

7 

1 

100 

0 

100 

5 

98 

1 

100 

0 

100 

8 

0 

100 

0 

100 

2 

100 

0 

100 

0 

100 

9 

0 

100 

0 

100 

0 

100 

0 

100 

0 

100 

Table  B.7 


AMI  NORRIS  CORONARY  PROGNOSTIC  INDEX 
(pre-  and  post-PPS  sample  combined) 


%  of  Patients 
Code       Score  (n=2853) 


Age 
>=  80 
70-79 
<  70 


3.9 
3.1 
2.3 


26.9 
49.1 
23.9 


MI   location/extent  (1) 


MD 
MD 
MD 
MD 

MD: 
EKG 
EKG 
EKG 


EKG : 
EKG : 


anterior  &  inferior 

anterior 

inferior 

transmural  but 

location  unspecified 

subendocardial 

anterior 

inferior 

transmural  but 

location  unspecified 

subendocardial 

no  evidence  of  MI 


no  EKG 


-1. 
2 
3 
4 

5 
6 
7 

8 

9 
10 
M 


3.0 
2.8 

2.0 
2.0 

0.  84 
2.8 


0.84 
0 


1.1 
27.3 
24  .  9 

9.3 


16. 

1. 
1. 
0. 

1 . 

12. 
5. 


Systolic  blood  pressure  on  admission 

<  55  1  10  0.5 

55-64  2  7  0.7 

65-74  3  6  0.7 

75-84  4  5  2.1 

85-94  5  4  2.9 

95-104  6  3  4.9 

105-114  7  2  7.5 

115-124  8  1  10.5 

>-  125  9  0  69.8 

No  measurement  on  admission                          M  0  0.5 


Cardiomegaly  by  CXR 
Yes 
No 

No  CXR  D1H  or  D2H 

Pulmonary  Congestion 
Severe  CHF  by  CXR 
Moderate  CHF  by  CXR 
Mild  CHF  by  CXR 
Lungs  not  clear. MD  exam 

and  unlikely  or  no 

CHF  by  CXR 


1  1.1  43.8 

0  0  48.2 
M  0  8.0 

1  3.3  8.1 

2  2.0  9.3 

3  1.0  17.7 

4  1.0  24.0 


History  of  Ischemia 

Prior  angina  or  1  0.4  46.7 

prior  MI 


AMI:     3/10/89,    11:49,  JK 


(1)     Defined  by  an  MD  review.     If  the  MD  did  not  identify  an  MI 
(code3  1-5) ,   the  MI  location  as  described  on  the  EKGs  report  on  days 
1  or  2  was  used. 
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Table  B.8  shows  the  distribution  of  patients  by  the  variables  used  in  the  Killip  Scale  for 
categorizing  patients  with  acute  myocardial  infarction  for  the  pre-  and  post-PPS  sample  com- 
bined (Killip  and  Kimball,  1967). 

Table  B.9  shows  the  distribution  of  patients  by  variables  used  in  RAND's  mental  status 
measure  for  patients  with  cerebrovascular  accident  by  frequency  and  mortality  rates.  The 
RAND  clinical  weights  are  based  upon  clinician  judgment. 

Table  B.10  compares  the  distribution  of  patients  by  RAND's  mental  status  measure  (as 
defined  in  Table  B.9)  and  the  APACHE  APS  mental  status  measure  for  the  pre-  and  post- 
PPS  sample  combined.  This  allows  one  to  see  the  percentage  distribution  of  patients  by  the 
more  detailed  categories  of  the  RAND  measure  and  the  less  detailed  categories  of  RAND's 
interpretation  of  the  APACHE  measures. 

Table  B.ll  shows  the  distribution  and  mortality  rates  for  patients  by  variables  used  in 
the  Goldman  Cardiac  Risk  Index  for  the  pre-  and  post-PPS  hip  fracture  sample  combined 
(Goldman  et  al.,  1977). 

Table  B.12  shows  the  distribution  of  APACHE  II  APS  variables  for  five  diseases  in  the 
pre-  and  post-PPS  sample.  The  codes  refer  to  variable  categories  used  in  the  PPS  data  set. 
The  scores  refer  to  the  number  of  points  assigned  for  patient  values  in  each  category.  The 
scores  presented  were  developed  by  Knaus  et  al.  (1985). 

Table  B.13  shows  the  distribution  of  patients  by  RAND  Acute  Complexity  (sickness-at- 
admission)  variables  for  the  pre-  versus  post-PPS  sample.  These  disease-specific  lists  of  vari- 
ables are  designed  to  measure  acute  morbid  and  comorbid  conditions  present  at  the  time  of 
hospital  admission.  Abbreviations  used  in  Table  B.13  are  listed  below: 


Table  B.14  shows  the  distribution  of  patients  by  RAND  Chronic  Complexity  (sickness- 
at-admission)  variables  for  the  pre-  versus  post-PPS  sample.  The  disease-specific  lists  of 


AF 
CPK 


Atrial  fibrillation 
Creatine  phosphokinase 


Table  B.8 


AMI:      KILLIP  MI  SEVERITY 
(pre-  and  post-PPS  sample  combined) 


%  of  Patients 
(n=2853) 


MD  identifies  shock  D2H 
or  SBP  <   90  D1H  or  D2H 


25.7 


Severe  CHF  by  CXR 
D1H  or  D2H 


6.8 


Moderate 
D1H  or 


or  mild  CHF  by  CXR 
D2H 


19.9 


Unlikely 
D1H  or 


or  no  CHF  by  CXR 
D2H 


42.2 
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Table  B.9 

RAND'S  MENTAL   STATUS  MEASURE  FOR  PATIENTS  WITH  CEREBROVASCULAR 


ACCIDENT  BY 
(pre-  and 

FREQUENCY 
post-PPS 

AND  MORTALITY  RATES 
sample  combined) 

Patient's  Mental  Status 
Described  in  the 
Medical  Record 

RAND 
Clinical 
Weight 

Frequency 
of  Patients 

n  % 

In-Hospital 
Mortality 
(%  Dead) 

No  mental  status  problem 
problem  identified  by- 
physician  or  nurse 

0 

937  33.2 

4.3 

Confusion  noted  by 
physician  or  nurse 

1 

448  15.9 

8.0 

Change  in  neurologic 
status  or  patient 
unable  to  follow 
commands  noted  by 
physician  or  nurse 

2 

702  24.9 

16.4 

Coma  noted  by  physician 
or  nurse 

3 

301  10.7 

32.2 

Coma  noted  by  physician 
and  nurse 

4 

224  7.9 

60.7 

Posturing  or  no  response 
to  pain  noted  by  physician 

5 

212  7.5 

67.9 

Total 

2824  100.0 

20.1 
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Table  B.ll 

GOLDMAN  CARDIAC  RISK  INDEX 
(pre-  and  post-PPS  sample  combined;  n  =  2762) 


Distribution 
of  Patients 


n 


%  Mortality 


Goldman  score  day  1,  2 
(all  patients,    regardless  of  surgery) 

0-9 

10-16 

>=  17 

All 


1735 
802 
225 

2762 


62.8 
29.0 
8.2 
100 


3.0 
6.4 
7.6 
4.4 


Goldman  score  day  1,  2 
(only  patients  with  surgery) 

0-9 

10-16 

>=  17 

All 


1634 
747 
206 

2587 


63.1 
28.9 
8.0 
100 


2.8 
4.8 
5.8 
3.6 


Goldman  score  day  1,  2 
(only  patients  with  no  surgery) 

0-9 

10-16 

>=  17 

All 


101 
55 
19 

175 


57  .  7 
31.4 
10.9 
100 


6.9 
27.3 
26.3 
15.4 


Goldman  score  immediately  before  surgery 
(only  patients  with  surgery) 

0-9 

10-16 

>=  17 

All 


1682 
714 
191 

2587 


65.  0 
27.6 
7.4 
100 


3.0 
4.5 
6.3 
3.6 
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variables  are  designed  to  measure  chronic  morbid  and  comorbid  conditions  present  at  the 
time  of  hospital  admission.  Abbreviations  used  in  Table  B.14  are  listed  below: 

Atrial  Fib        Atrial  fibrillation 

ESR  Erythrocyte  sedimentation  rate 

Table  B.15  shows  the  distribution  of  patients  by  Body  System  variables  for  the  pre- 
versus  post-PPS  sample.  The  disease-specific  definitions  of  each  variable  are  listed  in  the 
table. 

SGOT  Serum  Glutamic  Oxalacetic 

Transaminase  (ALT) 
WBC  White  blood  cell 

Table  B.16  shows  the  distribution  of  patients  by  the  number  of  body  systems  with  sick- 
ness at  admission  for  the  pre-  versus  post-PPS  sample.  Body  system  sickness  at  admission  is 
defined  in  Table  B.5.  Both  percentages  and  cumulative  (cum)  percentages  are  listed. 

Table  B.17  shows  the  distribution  of  patients  by  the  variables  used  in  the  Norris 
Coronary  Prognostic  Index  (Norris  et  al.,  1969).  The  codes  refer  to  variable  categories  used 
in  the  PPS  data  set.  The  scores  refer  to  the  number  of  points  assigned  for  patient  values  in 
each  category.  The  scores  presented  were  developed  by  Norris.  Electrocardiograms  from  day 
1  of  the  hospitalization  were  used  to  assign  location  and  extent  of  the  infarct.  If  no  day  1 
electrocardiogram  was  available,  the  day  2  electrocardiograms  were  used. 

Table  B.18  shows  the  distribution  of  patients  by  the  variables  used  in  the  Killip  Scale 
for  Categorizing  Patients  with  Acute  Myocardial  Infarction  for  the  pre-  versus  post-PPS  sam- 
ple (Killip  and  Kimball,  1967). 

Table  B.19  shows  the  distribution  and  mortality  rates  for  patients  by  variables  used  in 
the  Goldman  Cardiac  Risk  Index  for  the  pre-  versus  post-PPS  hip  fracture  sample  (Goldman 
et  al.,  1977). 
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Table  B.17 


AMI   NORRIS   CORONARY  PROGNOSTIC  INDEX 
(pre-PPS  compared  with  post-PPS) 


%  of  Pre         %  of  Post 
Patients  Patients 
Code       Score  (n=1416)  (n=1437) 


Age 


>=  80 

1 

3 

9 

25 

7 

28 

.1 

70-79 

2 

3 

1 

48 

2 

50 

.0 

<  70 

3 

2 

3 

26 

1 

21 

.9 

location/extent  (1) 

MD  : 

anterior  &  inferior 

1 

3 

0 

0 

8 

1 

.3 

MD  : 

anterior 

2 

2 

8 

27 

8 

26 

.9 

MD: 

inferior 

3 

2 

0 

25 

6 

24 

.  1 

MD : 

transmural  but 

4 

2 

0 

9 

2 

9 

.3 

location  unspecified 

MD: 

subendocardial 

5 

0 

84 

14 

3 

17 

.8 

EKG: 

anterior 

6 

2 

8 

1 

2 

1 

.1 

EKG: 

inferior 

7 

2 

0 

0 

9 

1 

.1 

EKG: 

transmural  but 

8 

2 

0 

0 

1 

0 

.  1 

location  unspecified 

EKG:     subendocardial  9  0.84  1.0  1.1 

EKG:     no  evidence  of  MI  10  0  14.0  11.8 

No  EKG  M  0  5.0  5.4 


Systolic  blood  pressure  on  admission 
<  55 
55-64 
65-74 
75-84 
85-94 
95-104 
105-114 
115-124 
>=  125 

No  measurement  on  admission 

Cardiomegaly  by  CXR 
Yes 
No 

No  CXR  D1H  or  D2H 

Pulmonary  Congestion 
Severe  CHF  by  CXR 
Moderate  \027F  by  CXR 
Mild  CHF  by  CXR 
Lungs  not  clear. MD  exam 

and  unlikely  or  no 

CHF  by  CXR 

History  of  Ischemia 
Prior  angina  or 
prior  MI 


AMI:      3/10/89,    11:49,  JK 
(1)     Defined  by  an  MD  review.     If  the  MD  did  not  identify  an  MI  (codes 
1-5),   the  MI  location  as  described  on  the  EKGs  report  on  days  1  or  2  was 
used . 


1 
2 
3 
4 
5 
6 
7 
8 
9 

M 


10 
7 
6 
5 
4 
3 
2 
1 
0 
0 


0. 
0, 
0. 
1 . 

3, 
4. 
7  . 
9. 
71 . 
0  , 


0.6 
0.7 
0  .  8 
2  .  5 
2  .  7 
5.2 
7  .  9 
11.1 
68.1 
0  .  4 


1 
0 
M 


1  .  1 

0 


41.9 
49.5 
8.6 


45. 

47  . 
7  . 


7.5 
8.8 
18.5 
23.4 


8.7 
9.9 
17  .  0 
24.6 


0  .  4 


44  .  6 


48.8 
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Table  B.18 

AMI:      KILLIP  MI  SEVERITY 
(pre-PPS  compared  with  post-PPS) 


of  Pre-PPS  %  of  Post-PPS 

Patients  Patients 
(n=1416)  (n=1437) 


MD  identifies  shock  D2H  22.7  28.5 

or  SBP  <  90  D1H  or  D2H 

Severe  CHF  by  CXR  6.0  7.5 
D1H  or  D2H 

Moderate  or  mild  CHF  by  CXR  20.8  19.0 
D1H  or  D2H 

Unlikely  or  no  CHF  by  CXR  44.0                                     4  0.4 
D1H  or  D2H 
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Appendix  C 


FURTHER  DETAILS  ON  MEASURING  EXPLICIT 
PROCESS  OF  CARE 


Appendix  C  provides  background  material  about  the  model  used  for  developing  explicit 
process  criteria  and  about  how  explicit  process  measures  were  analyzed.  The  appendix  also 
lists  the  disease-specific  explicit  process  criteria  for  each  disease  and  the  compliance  found 
with  these  criteria,  compares  the  performance  on  process  criteria  across  diseases  for  the  pre- 
and  post-PPS  sample  combined,  compares  the  performance  on  process  criteria  across 
diseases  for  the  pre-PPS  versus  the  post-PPS  sample,  provides  background  material  about 
the  model  used  for  developing  scales  from  the  explicit  process  criteria,  and  lists  the  perfor- 
mance for  each  disease  on  process  scales. 

DEVELOPING  PROCESS  CRITERIA 

Process  of  care  refers  to  medical  care  activities  delivered  to  patients  by  physicians,  nurses, 
or  technicians.  Hospital-based  processes  of  care  can  be  divided  into  diagnostic  and  therapeutic 
categories.  Each  of  these  can  be  further  divided  into  cognitive  and  technical  processes. 

The  function  of  diagnostic  processes  is  to  accumulate  more  data  about  the  patient  in  an 
attempt  to  develop  a  precise  therapeutic  plan  that  would  improve  patient  outcome.  Exam- 
ples of  diagnostic  cognitive  processes  include  physician  and/or  nurse  assessment  of  whether 
the  patient  has  shortness  of  breath  or  confusion.  Diagnostic  technical  processes  include 
ordering  X-rays,  electrocardiograms,  and  laboratory  studies.  Placing  patients  in  the  ICU  is 
diagnostic  in  that  ICU  placement  increases  the  number  of  physician  and  nurse  observations 
per  patient.  It  may  be  considered  both  a  cognitive  and  technical  process. 

Therapeutic  processes  are  designed  to  improve  patients'  experiences  and  outcomes. 
Therapeutic  cognitive  processes  in  hospital-based  acute  care  medicine  include  patient  coun- 
seling and  physical  therapy.  Therapeutic  technical  processes  include  use  of  oxygen  therapy, 
medications,  and  surgery. 

Each  process  can  occur  on  any  hospital  day  and  the  appropriateness  of  the  process  can 
and  should  be  assessed  in  relation  to  when  it  was  performed.  For  example,  one  can  assess 
the  use  of  telemetry  or  the  ICU  in  the  initial,  mid,  or  terminal  part  of  the  hospital  stay.  Pro- 
cess measures  should  be  performed  in  a  timely  fashion  depending  upon  the  urgency  of  the 
patient's  need.  Some  process  measures  must  be  performed  on  a  certain  day  to  be  appropri- 
ate; other  process  measures  can  take  place  at  any  time  during  the  hospital  stay.  Table  C.l 
shows  the  way  process  can  be  measured  throughout  the  days  of  the  hospital  stay  for  diagnos- 
tic and  therapeutic  services. 

All  process  measures  are  conditional  if-then  statements.  We  can  consider  measures 
applicable  to  all  patients,  conditional  on  patients  being  hospitalized  with  that  disease.  For 
example,  all  patients  hospitalized  with  congestive  heart  failure  should  have  a  physician  lung 
exam  on  each  day.  Measures  applicable  to  subsets  of  patients  must  define  the  subset  in  the 
conditional  statement.  For  example,  only  very  sick  patients  should  use  the  ICU  or  receive 
pressors. 
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Developing  Abstraction  Forms 

To  measure  quality  of  care,  we  used  a  number  of  criteria  for  developing  process  mea- 
sures. First,  we  sought  process  measures  that  could  be  reliably  studied  using  the  medical 
record  as  a  data  base.  To  assess  whether  suggested  process  measures  were  reliably  docu- 
mented in  the  medical  record,  we  pilot  tested  process  measures  using  50-75  medical  records 
for  each  disease.  The  records  were  selected  from  1981  to  1986,  from  a  mixture  of  hospital 
types  in  each  of  the  five  study  states.  We  studied  whether  both  the  if  and  the  then  com- 
ponents of  the  process  measures  were  reliably  found  within  the  record. 

For  example,  our  pilot  data  demonstrated  that  items  such  as  use  of  ICU  or  medications 
were  reliably  recognizable  from  the  record.  In  contrast,  the  number  of  times  a  frail  patient 
was  moved  from  the  bed  to  a  chair  for  upright  posture  and  conditioning  was  not  easily  mea- 
sured from  the  record  and  was  not  included  as  a  process  measure.  The  time  it  took  for  a 
nurse  to  respond  to  a  bedridden  patient's  call  and  the  adequacy  of  nutrition  were  also  impor- 
tant. However,  these  processes  are  difficult  to  measure  using  the  medical  record  and  we  did 
not  assess  them. 

Second,  we  sought  process  measures  that  were  clinically  valid.  We  began  this  process 
with  literature  review  and  consultation  with  subspecialist  and  generalist  experts.  We 
specifically  asked  the  experts  to  define  process  measures  they  believe  make  a  difference  in 
patient  outcome.  After  listing  such  process  measures,  the  research  team  selected  a  subset 
that  were  likely  to  be  measurable  from  the  medical  record.  These  measures  were  presented 
to  disease-specific  panels  consisting  of  5-12  PRO-selected  physicians.  Table  C.2  shows  physi- 
cian specialty  types  selected  for  each  of  the  study  diseases. 

At  these  panel  meetings,  RAND  physician  researchers,  1-2  UCLA  subspecialty  consul- 
tants, and  PRO  physicians  reviewed  the  suggested  criteria  to  evaluate  whether  as  a  group 
they  believed  these  criteria  were  likely  to  be  reliably  recorded  in  the  medical  record  and 
whether  the  process  criteria  made  clinical  sense.  Specifically,  we  asked  how  important  they 
thought  each  measure  was  as  a  process  criterion.  We  paid  particular  attention  to  physicians' 
comments  that  suggested  variations  in  recording  of  process  by  year,  state,  and/or  hospital 
type.  We  finally  included  as  process  measures  only  those  that  the  physician  panel  believed 
were  reliably  recorded  and  clinically  important  in  determining  outcomes.   We  selected 

Table  C.2 

DISEASE-SPECIFIC  PHYSICIAN  SPECIALTY  PANELS 


Number 
of  Physician 


Disease 

Physician  Specialty 

Participants 

Acute  myocardial  infarction 

Cardiology 

12 

Congestive  heart  failure 

Internal  medicine 

12 

Hip  fracture 

Orthopedics 

7 

Pneumonia 

Pulmonology 

5 

Cerebrovascular  accident 

Neurology 

5 

Depression 

Psychiatry 

5 
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process  measures  that  the  panelists  believed  could  discriminate  excellent,  good,  adequate, 
poor,  and  very  poor  medical  care  in  any  area  of  the  country  from  1981  through  1986. 

Third,  we  selected  process  measures  that  involved  reasonably  stable  technologies  from 
1981  to  1986.  We  did  not  develop  process  criteria  that  would  penalize  1981  process  scores  for 
using  technologies  such  as  angioplasty  or  thrombolytic  therapy  that  were  less  available  dur- 
ing the  early  part  of  the  decade. 

Fourth,  we  collected  items  that  could  be  abstracted  from  the  medical  record  within  a 
few  minutes,  since  the  amount  of  time  for  complete  record  abstraction  was  fixed  at  an  aver- 
age of  75  minutes  per  record  for  this  study.  Given  the  fixed  time  for  abstraction,  we  did  not 
attempt  to  measure  every  inhospital  process.  Instead,  we  selected  process  measures  to 
represent  categories  of  process.  We  developed  some  process  measures  that  would  apply  to  all 
diseases.  In  addition,  for  each  disease  we  studied  some  aspect  of  process  in  more  detail  than 
for  other  diseases.  Table  C.3  shows  categories  of  process  studied  for  each  disease. 

Fifth,  we  developed  process  measures  that  were  applicable  to  a  large  number  of 
patients,  since  the  cost  of  data  collection  is  significant. 


Table  C.3 

CATEGORIES  OF  PROCESS  MEASURES  FOR  EACH  DISEASE 


Use  of  Services 


CHF     AMI     HIP     PNE     CVA  DEP 


Diagnostic  services 

MD  cognitive  assessment 
RD  cognitive  assessment 
Venous  blood  tests 
Arterial  blood  tests 
X-ray 
EKG 

Microbiology 
Pathology 

Cardiac  monitoring 
Monitoring  procedures 

Therapeutic  services 
Meds 
Oxygen 

Ancillary  services 

(e.g.,  physical  respiratory- 
therapy) 

RX  procedures 

(e.g.,    surgery  electro- 
convulsive therapy) 


X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 
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The  final  set  of  process  data  collected  from  the  medical  record  was  based  on  views  of  the 
physician  panelists  and  those  of  the  research  team.  We  combined  clinical  judgment  with 
expertise  in  what  is  measurable  from  the  medical  record.  The  instruments  used  for  measur- 
ing process  have  been  published  elsewhere  (Kahn  et  al.,  1988b;  Kosecoff  et  al.,  1988;  Roth  et 
al.,  1988;  Rubenstein  et  al,  1988;  Sherwood  et  al,  1988;  Wells  et  al.,  1988). 

Defining  Process  Measures  for  Subsets  of  Patients 

Using  the  disease-specific  panel  meetings,  we  collected  data  about  patients'  conditions 
(or  sickness  levels)  throughout  the  hospital  stay.  In  addition,  we  collected  data  about  use  of 
services.  Each  process  measure  was  designed  to  be  specific  for  a  patient's  sickness.  The  pro- 
cess criteria  do  not  necessarily  consider  more  process  as  better.  Instead,  the  criteria  rate 
more  appropriate  process  as  better.  For  example,  use  of  antibiotics  is  considered  good  care 
only  if  a  clear  therapeutic  or  prophylactic  indication  exists. 

To  develop  clinically  valid  process  measures,  both  the  if  and  the  then  part  of  the  process 
statement  must  be  defined.  To  define  the  i/*part  of  the  conditional  process  statement,  we  are 
obligated  to  define  the  group  of  sick  patients  for  whom  specific  interventions  are  appropriate. 
Then  conditional  statements  measure  whether  a  process  was  or  was  not  performed.  When  if 
can  be  precisely  defined  (e.g.,  bacteremia)  and  then  (e.g.,  use  of  appropriate  antibiotics) 
improves  outcome,  the  conditional  if-then  statement  is  generally  accepted  in  clinical  practice 
and  can  provide  the  basis  for  measuring  quality  of  care.  In  practice,  clinicians  are  sometimes 
faced  with  patients  with  single  and  specific  problems  (e.g.,  bacteremia)  and  can  respond  to 
simple  process  criteria  such  as,  "if  bacteremia,  then  use  antibiotics."  More  often,  however, 
during  hospital-based  acute  care  medical  practice  (especially  for  the  elderly)  clinicians  are 
faced  with  a  combination  of  symptoms  and  signs  that  characterize  the  patient  as  sick. 

Clinicians  recognize  at  the  bedside  the  patient  who  requires  diagnostic  and/or  therapeu- 
tic interventions  because  that  patient  is  generally  sick  and  unstable.  The  clinician  gauges 
the  type,  number,  and  timing  of  interventions  (then  statements)  according  to  the  severity  of 
the  patient's  sickness  (if  statements).  As  the  number  of  abnormal  symptoms  and  signs 
increases,  the  clinician  generally  becomes  increasingly  concerned  with  the  patient  and  gen- 
erally is  obligated  to  make  more  diagnostic  and  therapeutic  interventions.  For  example,  if  a 
patient  shows  new  distress  (e.g.,  shortness  of  breath  or  chest  pain)  or  unstable  vital  signs 
(e.g.,  tachycardia  or  tachypnea)  the  clinician  recognizes  that  an  intervention  should  be  made. 
When  the  clinician  notes  a  combination  of  these  problems,  the  clinician  generally  makes  the 
interventions  more  urgent  and  comprehensive.  If  the  problem  is  new,  the  clinician  is  even 
more  aggressive  with  diagnostic  and  therapeutic  interventions. 

Defining  Sickness  for  Subsets  of  Patients 

In  designing  a  system  for  measuring  quality  of  care  for  sick  patients,  we  faced  the  chal- 
lenge of  objectively  defining  sickness  for  the  if  part  of  conditional  if-then  statements.  Should 
sickness  be  defined  by  symptoms,  signs,  or  both?  Which  symptoms  or  signs  are  the  best  mea- 
sure of  sickness?  How  do  symptoms  and  signs  combine  to  determine  degrees  of  sickness? 
Does  the  importance  of  symptoms  and  signs  vary  by  disease?  To  answer  these  questions  we 
developed  a  new  method  for  defining  patient  sickness  and  validated  it  clinically. 

For  each  patient  hospitalized  with  congestive  heart  failure,  acute  myocardial  infarction, 
hip  fracture,  pneumonia,  and  stroke,  we  collected  data  about  some  or  all  of  the  following  vari- 
ables: shortness  of  breath  (presence  or  absence),  chest  pain  (presence  or  absence),  confusion 
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(presence  or  absence),  heart  rate  &  130,  respiratory  rate  a  30,  systolic  blood  pressure  <  90, 
and  diastolic  blood  pressure  a  105.  We  considered  patients  as  having  shortness  of  breath, 
chest  pain,  or  confusion  if  the  physician  and/or  nurse  described  the  symptom  in  the  record  for 
the  day  under  examination.  Vital  signs  were  abstracted  from  graphic  sheets,  ICU  flow 
sheets,  and  nurses  notes. 

For  congestive  heart  failure,  acute  myocardial  infarction,  hip  fracture,  pneumonia,  and 
cerebrovascular  accident  we  collected  data  about  heart  rate,  respiratory  rate,  systolic  blood 
pressure,  and  diastolic  blood  pressure. 

For  congestive  heart  failure  and  pneumonia,  we  had  a  trichotomous  variable  for  short- 
ness of  breath.  That  allowed  us  to  distinguish  the  presence  of  shortness  of  breath  from  its 
absence,  and  from  no  data  for  each  hospital  day  studied.  For  acute  myocardial  infarction  we 
used  a  trichotomous  variable  for  chest  pain  that  allowed  us  to  distinguish  the  presence  and 
absence  of  chest  pain  from  no  data  about  chest  pain  that  day.  All  other  variables  were 
dichotomous  (yes  versus  no/no  data).  We  considered  patients  confused  only  if  they  did  not 
have  a  history  of  dementia,  except  for  hospital  day  1  on  which  dementia  was  not  considered. 

For  cerebrovascular  accident,  we  collected  data  about  the  same  variables  listed  above 
except  for  chest  pain  and  confusion.  For  hip  fracture,  we  collected  data  only  about  signs,  not 
about  symptoms.  For  each  variable,  we  asked  data  collectors  to  indicate  the  patients'  worst 
status  for  the  day.  For  example,  if  the  systolic  blood  pressure  was  <  90  on  only  one  of  several 
readings  that  day,  we  considered  systolic  <  90  that  day. 

We  used  logistic  regression  on  our  dataset  to  determine  the  value  of  each  of  the  symp- 
toms and  signs  in  predicting  inhospital  mortality.  For  congestive  heart  failure  and  acute 
myocardial  infarction,  we  validated  the  logistic  weights  on  a  separate  dataset  of  1,500 
patients  for  each  disease  collected  for  the  Nonintrusive  Outcome  Study  (NOS).  The  NOS 
sample  was  designed  to  have  a  50  percent  mortality.  NOS  oversampling  on  mortality  dif- 
fered from  the  current  study  sample  that  randomly  selected  patients  based  on  diagnosis, 
regardless  of  outcome. 

After  studying  the  relationship  between  abnormal  symptoms  and  signs  and  inhospital 
mortality,  we  developed  a  sickness  scale  that  categorized  patients  according  to  the  presence 
or  absence  of  a  combination  of  symptoms  and  signs.  We  named  the  scale  ASSC,  the  acronym 
for  Any  Symptom  or  Sign  Complex.  Patients  were  assigned  increasing  ASSC  points  in  rela- 
tion to  the  number  and  type  of  each  of  the  following  symptoms  and  signs:  shortness  of 
breath,  chest  pain,  confusion,  heart  rate  a  130,  respiratory  rate  a  30,  systolic  blood  pressure  < 
90,  and/or  diastolic  blood  pressure  &  105.  Table  C.4  shows  the  relationship  of  ASSC  scores  to 
death  for  each  disease. 

We  developed  a  series  of  process  criteria  denning  subsets  for  if  conditional  statements. 
When  possible,  we  used  specific  conditional  if  statements  (e.g.,  if  febrile;  if  life-threatening 
potassium  value).  To  identify  generally  sick  and  unstable  patients,  we  used  the  generic 
ASSC  measure  of  sickness.  Table  C.5  defines  our  clinical  definitions  of  ASSC  categories. 

Timing  of  Process  Measures 

We  categorized  process  measures  according  to  when,  during  the  hospital  stay,  the  cri- 
teria should  be  met.  We  defined  the  initial  hospital  stay  as  the  first  two  days.  We  studied 
day  3  processes  for  study  diseases  as  a  sample  of  subsequent  hospital  days.  For  all  six  study 
diseases  we  collected  data  on  the  day  before  discharge  and  day  of  discharge.  For  some  pro- 
cess measures  (e.g.,  fever  >  101°F  or  38.3°C),  we  collected  process  data  regardless  of  when 
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Table  C.4 

MORTALITY  RATES  STRATIFIED  BY  ASSC 
(in  percent) 


CHF 

AMI 

PNE 

CVA 

CHF 

AMI 

PNE 

CVA 

Initial 

Score 

Day  3 

Score 

ASSC  score 

0 

0 

0 

2 

9 

4 

5 

3 

11 

1 

6 

8 

3 

17 

5 

8 

4 

16 

2 

4 

11 

3 

28 

9 

19 

10 

37 

3 

10 

15 

9 

46 

12 

24 

17 

42 

4 

8 

19 

3 

54 

20 

24 

24 

54 

5 

16 

25 

20 

65 

29 

30 

31 

62 

6 

16 

34 

23 

76 

48 

39 

50 

41 

>7 

32 

57 

48 

90 

50 

68 

69 

44 

HIP 

HIP 

Initial  Score 

Day  3  Score 

ASC  score 

0 

3 

3 

1 

5 

4 

2 

7 

11 

>3 

17 

9 

during  the  hospital  stay  the  if  component  of  the  conditional  process  measure  occurred.  For 
example,  for  CHF  patients,  if  temperature  >  101°F  (38.3°C),  then  at  least  one  fever  work-up 
should  take  place  regardless  of  when  during  the  hospital  stay  the  fever  occurred. 

Because  of  the  cost  of  data  collection,  we  were  unable  to  collect  clinically  detailed  data 
about  every  process  measure  for  each  hospital  day.  Although  we  collected  data  about  use  of 
some  services  (e.g.,  ICU  and  telemetry)  for  every  hospital  day,  we  were  able  to  define  the 
need  for  those  services  only  for  a  sample  of  days.  We  elected  to  supplement  process  measures 
on  initial,  subsequent  (i.e.,  day  3),  and  discharge  days  with  process  measures  around  the  time 
a  patient  became  very  sick.  Specifically,  for  four  study  conditions  (CHF,  AMI,  HIP,  PNE)  we 
selected  a  group  of  very  sick  patients.  We  categorized  patients  as  very  sick  if  they  sustained 
an  adverse  outcome.  Disease-specific  adverse  outcomes  are  defined  in  Table  C.6.  We  then 
collected  clinically  detailed  data  from  the  medical  record  about  process  measures  delivered 
on  the  days  before  and  on  the  day  of  the  adverse  outcomes  for  these  very  sick  patients.  This 
data  collection  was  designed  to  assess  the  preventability  of  the  adverse  outcome.  We  also  col- 
lected data  about  the  patients'  status  during  the  days  following  the  adverse  outcome  to 
assess  whether  it  was  appropriately  diagnosed  and  treated.  For  CHF,  AMI,  and  PNE,  we 


230 


Table  C.5 
CLINICAL  DEFINITIONS  OF  ASSC 


Sickness  Category 

ASSC 

Score  Values 

CHF(l) 

AMI(l) 

HIP(2) 

PNE(l) 

CVA(3) 

Not  very  sick 

0-2 

0-2 

0-1 

0-2 

0 

Somewhat  sick 

3-4 

3-4 

2 

3-4 

1 

Moderately  sick 

5-6 

5-6 

>  3 

5-6 

2-3 

Very  sick 

>  7 

>  7 

>  7 

>  4 

ASSC  variables  for  CHF,  AMI,  and  PNE  include:  shortness 
of  breath  (1),  chest  pain  (1),  confusion  (2),  heart  rate  > 
130  (2),   respiratory  rate  >  30  (2),  systolic  <  90  (3), 
diastolic  >  105   (1).     Numbers  in  parentheses  are  the  points 
assigned  for  the  presence  of  each  variable. 


Variables  for  HIP  include:     heart  rate  >  130  (2), 
respiratory  rate  >  30  (2),  systolic  <  90  (3),  diastolic  >  105 
(1).     Numbers  in  parentheses  are  the  points  assigned  for  the 
presence  of  each  variable. 

3 

Variables  for  CVA  include:     shortness  of  breath  (1),  heart 
rate  >  130  (2),   respiratory  rate  >  30  (2),  systolic  <  90  (3), 
diastolic  >  105   (1).     Numbers  in  parentheses  are  the  points 
assigned  for  the  presence  of  each  variable. 


studied  the  five  days  surrounding  each  patient's  first  adverse  outcome.  For  HIP,  we  studied 
the  day  before  and  day  of  each  patient's  first  adverse  outcome. 

Table  C.7  demonstrates  the  time  periods  for  process-related  data  collection  for  each  of 
six  study  diseases. 

ANALYZING  PROCESS  MEASURES 

We  used  clinical  judgment  to  assign  points  to  each  process  measure  according  to  how 
important  we  believed  each  process  was  in  determining  outcomes.  The  basis  for  clinical  judg- 
ment was  literature  review  and  clinical  experience.  In  assigning  points,  the  clinician 
researchers  were  blind  to  differences  in  the  use  of  services  during  the  pre-  and  post-PPS 
periods. 

The  major  challenge  in  scoring  process  measures  was  to  assign  "good"  process  points 
only  for  the  meeting  of  clinically  important  and  relevant  process  criteria.  The  use  of  more 
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Table  C.6 
DEFINITIONS  OF  ADVERSE  OUTCOMES 


Definitions  of 

Adverse  Outcomes  CHF        AMI        HIP        PNE        CVA(l)  DEP(l) 


Seizure,  syncope,  or 
loss  of  consciousness 


on  day  2  or  later 

X 

X 

X 

Shock  on  day  2  or  later 

X 

X 

X 

X 

Coma  on  day  2  or  later 

X 

X 

X 

Arrest   (except  in  ER) 

X 

X 

X 

X 

Death 

X 

X 

X 

X 

Recurrent  chest  pain 
day  3  or  later 
(on  >  2  consecutive  days) 

X 

Endotracheal  or  nasotracheal 
intubation  (except  in  ER) 

X 

X 

For  cerebrovascular  accident  and  depression,  process  measures  were 
not  specifically  collected  around  the  adverse  outcome. 


services  should  earn  more  process  points,  only  if  those  additional  services  were  clinically 
indicated. 

We  attempted  to  develop  a  scoring  system  that  would  score  only  appropriate  process 
and  that  would  take  account  of  the  number  and  importance  of  process  measures  applicable  to 
each  patient  according  to  their  level  of  sickness. 

ASSIGNING  PROCESS  POINTS  FOR  PROCESS  CRITERIA 

For  every  process  criterion,  we  used  clinical  judgment  to  determine  the  maximum 
number  of  points  that  could  be  achieved  when  the  patient  was  optimally  treated.  This  was 
the  number  of  "applicable"  points  for  that  criterion.  For  example,  for  a  very  sick  patient  (e.g., 
ASSC  a  7)  who  should  be  in  the  ICU  on  a  particular  day,  we  assigned  7  "applicable"  or  poten- 
tial points  for  the  process  criteria  related  to  ICU  use  that  day. 

We  also  designated  "assigned"  process  points  to  characterize  care  that  was  actually 
given  to  the  patient.  If  the  patients  received  optimal  care,  then  their  "assigned"  points  would 
equal  their  "applicable"  points.  For  example,  the  patient  monitored  in  the  ICU  would  receive 
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Table  C.7 


HOSPITAL  DAYS  WHEN  PROCESS  DATA  WERE  COLLECTED 


CHF 


AMI 


HIP 


PNE 


CVA 


DEP 


Initial  days 

(i.e. ,  day  1  and  2) 

Day  3 

Days  around  first 

adverse  outcome (1) 

Days  around  surgery 

Discharge  days 

(i.e.,  day  before  and 
day  of  discharge) 

First  time  problem 
occurs  regardless 
of  day  (i.e.  life 
threatening  lab, 
fever) 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


For  CHF,  AMI,  and  PNE  we  studied  two  days  before  the  adverse 

outcome  day,  the  adverse  outcome  day,   and  two  days  after  the  adverse 

outcome  day.  For  HIP  we  studied  the  day  before  and  day  of  the  adverse 
outcome . 


7  "assigned"  points  from  a  possible  7  "applicable"  points  and  receive  100  percent  of  the  possi- 
ble process  points  for  monitoring  that  day. 

Since  patients  did  not  always  receive  optimal  care,  we  used  clinical  judgment  to  assign 
points  for  less  than  optimal  strategies.  For  example,  for  CHF,  if  telemetry  rather  than  ICU 
were  used  to  monitor  a  very  sick  patient  (e.g.,  ASSC  a  7),  then  only  3  "assigned"  of  a  possible 
7  "applicable"  points  would  be  given  (43  percent).  If  the  patients  received  neither  telemetry 
nor  ICU  monitoring,  then  they  would  receive  0  "assigned"  of  7  "applicable"  ICU  monitoring 
points  (0  percent). 

In  developing  scores  for  process  criteria,  we  attempted  to  maintain  consistency  across 
diseases  and  within  diseases.  For  example,  if  a  patient  were  very  sick  and  would  likely 
benefit  from  ICU  monitoring,  we  assigned  7  points  across  diseases.  We  assigned  1  point  for 
each  blood  chemistry  test  across  diseases.  We  assigned  2  points  for  ordering  an  electrocar- 
diogram to  rule  out  a  myocardial  infarction  and  assess  rhythm  in  a  very  sick  patient  admit- 
ted with  CHF. 

We  adjusted  the  relative  number  of  applicable  points  assigned  according  to  how  impor- 
tant clinician  judgment  suggested  it  was  for  a  criterion  to  be  met  and  according  to  how  likely 
the  data  demonstrated  the  criterion  was  to  be  met.  Clinical  judgment  suggests  that  ICU 
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monitoring  is  important  for  sick  patients.  This  is  reflected  in  the  process  criteria:  If  sick,  the 
patient  should  have  ICU  monitoring. 

We  adjusted  scores  for  process  criteria  according  to  the  clinically  expected  benefit  of  using 
the  services  for  subsets  of  patients.  Table  C.8  shows  scores  for  use  of  ICU  and  telemetry  for  dif- 
ferent categories  of  CHF  patients  classified  by  sickness  (i.e.,  ASSC  score).  The  maximum 
number  of  "applicable"  points  drops  as  the  clinical  benefit  to  the  patient  is  expected  to  drop.  For 
very  sick  patients,  7  applicable  process  points  are  assigned;  this  number  of  applicable  points 
drops  to  3  for  moderately  sick  patients,  and  to  0  for  other  patients. 

The  scores  also  take  into  account  the  differential  value  of  ICU  versus  telemetry  for 
patients  with  different  sickness  levels.  Very  sick  patients  receive  42  percent  of  applicable 
points  (3  of  7)  for  using  telemetry,  whereas  moderately  sick  patients  receive  a  higher  fraction 
(67  percent)  of  applicable  points  for  telemetry  monitoring.  This  type  of  scoring  emphasizes 

Table  C.8 

ASSIGNED  AND  APPLICABLE  POINTS  FOR  ICU  AND  TELEMETRY  MONITORING 
FOR  CHF  PATIENTS  STRATIFIED  BY  SICKNESS 


Clinical 
ASSC 
Definition 


ASSC 
Score 


Type  of 
Monitoring 
Used 


Assigned 
Points 


Applicable 
Points 


% 

Assigned/ 
Applicable 
Points 


Very  sick 


ICU  or  CCU 

Telemetry 

None 


7 
3 
0 


7 
7 
7 


100 

0.42 

0.00 


Moderately  sick  5-6 


ICU  or  CCU 

Telemetry 

None 


3 
2 
0 


3 
3 
3 


1.00 
0.67 
0.00 


Somewhat  sick 


3-4 


ICU  or  CCU 

Telemetry 

None 


0 
0 
0 


0 
0 
0 


0.00 
0.00 
0.00 


Not  very  sick 


ICU  or  CCU 

Telemetry 

None 


0 
0 
0 


0 
0 
0 


0 
0 
0 


00 
00 
00 


NOTE:     CHF  process  criteria:     On  day  1  of  hospitalization  for  CHF  if 
the  patient  is  very  sick  (i.e.,  ASSC  >  =  7),  the  ICU  should  be  utilized. 
Telemetry  is  not  sufficient  but  is  preferable  to  no  cardiac  monitoring. 
On  day  1  of  hospitalization  for  CHF,  if  the  patient  is  moderately  sick 
(i.e.,  ASSC  5-6),  the  ICU  or  telemetry  should  be  used. 


234 


clinical  logic.  For  very  sick  patients  the  difference  between  ICU  use  and  telemetry  use  is 
valued  more  highly  than  the  difference  for  less  sick  patients.  Also  the  difference  between 
using  ICU  and  no  monitoring  is  greater  for  very  sick  than  for  less  sick  patients. 

For  this  and  other  process  measures  we  examined  the  empirical  frequency  with  which 
the  criteria  were  met  for  subsets  of  patients.  Table  C.9  shows  the  use  of  the  ICU  and 
telemetry  for  CHF  patients  according  to  level  of  sickness.  The  table  shows  that  67  percent  of 
very  sick  (i.e.,  ASSC  a  7)  CHF  patients  use  the  ICU  and  40-50  percent  of  moderately  sick 
(i.e.,  ASSC  5-6)  CHF  patients  used  the  ICU.  In  contrast,  only  23-39  percent  of  mildly  sick 
patients  (i.e.,  ASSC  3-4)  and  less  than  23  percent  of  not  very  sick  patients  (i.e.,  ASSC  s  3) 
used  the  ICU. 

We  further  validated  the  process  criteria  by  studying  mortality  rates  for  patient  sub- 
sets. If  the  mortality  increased  as  sickness  increased  (e.g.,  as  it  did  for  increasing  values  of 
ASSC — see  Table  C.4),  then  it  seemed  reasonable  to  expect  that  more  services  would  be  used 
to  diagnose  (e.g.,  with  laboratory  studies)  and  treat  (e.g.,  with  pressors)  the  patient. 

Table  CIO  discusses  the  power  of  the  study  design  to  detect  differences  in  care  pre- 
versus  post-PPS.  Clinically  meaningful  disease-specific  pre-  versus  post-PPS  differences  at 
the  level  of  a  single  process  item  that  apply  only  to  a  small  fraction  of  patients  may  not  show 
up  as  significant  in  our  data,  even  with  about  2,800  patients  per  disease  (Table  C.10). 

For  example,  a  pre-  versus  post-PPS  difference  in  percentage  of  patients  meeting  a 
disease-specific  criterion  that  applies  to  only  25  percent  of  patients  and  that  was  met  by  70 
percent  of  patients  in  the  pre-PPS  period  would  have  to  be  at  least  7  percentage  points  to 
show  up  as  significant  at  the  usual  5  percent  level,  i.e.,  a  pre-post  change  from  70  to  75  per- 
cent compliance  would  not  be  significant  for  such  an  item  but  a  change  from  70  percent  to  80 
percent  would  be  significant. 

For  this  reason,  we  did  not  draw  process  conclusions  at  an  item-specific  level.  Tables  like 
the  one  above  are  precisely  why  we  built  scales  consisting  of  weighted  sums  of  individual  items: 
The  scale  will  be  much  more  sensitive  than  any  of  the  items  making  it  up.  As  we  document  in 
Sec.  II,  if  our  final  composite  process  scale  is  rescaled  to  have  a  range  from  0  to  100  and  a  stan- 
dard deviation  of  10,  and  if  our  design  gave  us  at  least  80  percent  power  to  find  pre-  versus  post- 
PPS  differences  of  only  1 . 1  points  at  the  disease-specific  level,  we  had  power  to  detect  even  small 
process  differences  associated  with  the  introduction  of  PPS. 

The  main  negative  process  finding  was  our  inability  to  demonstrate  a  process-outcome 
link  for  hip  fracture,  with  outcome  measured  by  30-day  death  and  process  measured  by  our 
composite  scale.  We  again  do  not  believe  that  this  was  due  to  insufficient  power,  for  the  fol- 
lowing reason.  We  assessed  the  significance  of  our  process-outcome  links  in  each  disease  by 
logistically  regressing  30-day  death  on  the  composite  process  scale  and  our  scale  measuring 
sickness  at  admission,  and  examining  the  significance  of  the  process  coefficient.  It  is  difficult 
to  make  explicit  power  calculations  with  logistic  regression  coefficients,  but  a  close  approxi- 
mation can  be  obtained  by  examining  the  corresponding  coefficient  from  the  ordinary  least 
squares  regression  of  the  0-1  death  indicator  on  process  (for  simplicity  the  admission  sick- 
ness variable  is  omitted  from  these  calculations;  the  results  would  be  similar  if  it  were 
included).  It  is  straightforward  to  show  that  the  correlation  between  the  0-1  death  indicator 
y  and  the  composite  process  score  /  is 


Table  C.9 


DISTRIBUTION  OF  CHF  PATIENTS  BY  USE     OF  ICU  AND  TELEMETRY 
STRATIFIED  BY  ASSC  VALUES  ACROSS  HOSPITAL  DAYS 


ICU  or 

Hospital  ICU      Telemetry  Telemetry 

ASSC  Score  Day  n  %  %  % 


0  1  31  6  10  16 

1  1  726  11  21  32 

2  1  481  23  27  51 

3  1  555  23  22  46 

4  1  410  39  21  61 

5  1  263  40  15  56 

6  1  189  50  18  68 
>=  7  1  159  67  15  82 

0  3  977  17  21  38 

1  3  794  18  25  43 

2  3  329  26  24  50 

3  3  327  27  24  51 

4  3  107  35  27  62 

5  3  90  39  21  60 

6  3  44  52  16  68 
>=  7  3  48  69  12  81 

0  AO  29  11  30  41 

1  AO  27  12  16  28 

2  AO  47  36  13  49 

3  AO  82  21  23  44 

4  AO  44  33  36  69 

5  AO  66  32  21  52 

6  AO  56  50  13  63 
>=  7  AO  90  69  13  82 


AO    Day  of  Adverse  Outcome. 
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Table  C . 10 

MINIMUM  SIGNIFICANT  DISEASE-SPECIFIC  ITEM-LEVEL 
DIFFERENCES   IN  PROCESS   SCORES  PRE-  VERSUS  POST-PPS 

(In  percent) 


%  of 
Pre-PPS 

Patients  %  of  Patients  Applicable 

Meeting   

Criteria  100  75            50            25  10 

50  3.8  4.4           5.4           7.6  12.0 

40  or  60  3.7  4.3           5.2           7.4  11.7 

30  or  70  3.5  4.0           4.9           7.0  11.1 

20  or  80  3.0  3.5           4.2           6.0  9.5 

10  or  90  2.3  2.7           3.2           4.6  7.3 

5  or  95  1.6  1.8           2.3           3.2  5.0 


where  Xi  and  Xo  are  the  mean  process  scores  for  patients  who  died  and  lived,  respectively,  sx 
is  the  standard  deviation  of  the  process  scores,  and  p  is  the  fraction  of  patients  who  die 
within  30  days  of  admission  with  the  given  disease.  With  2,541  hip  fracture  patients  for 
whom  accurate  postdischarge  mortality  information  was  available,  the  smallest  such  correla- 
tion r  that  would  have  shown  up  as  significant  (i.e.,  2  standard  errors)  is  about  -0.04,  using 
Fisher's  approximate  transformation  to  normality 


1+r 
1-r 


For  hip  fracture  p  was  about  0.049,  so  that  the  process-outcome  link  would  have  shown  up  as 
significant  if  average  process  for  hip  fracture  patients  who  died  was  lower  than  average  pro- 
cess for  patients  who  lived  by  0.18  or  more  standard  deviations .  In  fact,  as  Table  C.ll  indi- 
cates for  hip  fracture  patients 


(Xi  "  Xo) 


=  +  0.08 


with  a  95  percent  confidence  interval  of  (-0.10  to  +  0.26).  The  data  do  not  exclude  a  small 
process-outcome  link,  which  we  might  have  found  with  a  large  sample.  This  result  might 
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also  have  been  caused  by  our  measurements  of  process  and  outcome.  A  better  outcome  for 
hip  fracture  would  probably  have  been  functional  status  several  months  after  discharge, 
which  we  were  not  able  to  measure.  Additionally  the  evaluation  of  process  for  surgical 
diseases  may  not  be  as  accurate  as  for  medical  diseases  because  of  limitations  of  data  in  the 
medical  record  (important  intra-operative  information  may  be  particularly  lacking,  for  exam- 
ple). As  we  discuss  in  Sec.  IV,  methods  for  better  evaluation  of  surgical  processes  of  care  are 
needed. 

EXPLICIT  PROCESS  CRITERIA  FOR  EACH  DISEASE 

Tables  C.12  through  C.16  list  the  explicit  process  criteria  for  each  of  the  five  study 
diseases.  The  tables  are  organized  into  four  columns.  The  columns  labeled  "if  specify  the 
group  of  patients  to  whom  the  criteria  are  applicable.  The  columns  labeled  "then"  specify  the 
criteria  to  be  met  for  the  relevant  subset  of  patients.  The  third  column  specifies  the  percent- 
age of  patients  to  whom  each  criterion  is  applicable.  For  example,  a  criterion  states,  "within 
the  initial  two  days  of  hospitalization,  the  physician  should  document  information  about 
medication  use  immediately  before  admission  for  all  patients  with  congestive  heart  failure." 

In  contrast,  the  criterion  of  ordering  a  cardiac  enzyme  study  (e.g.,  CPK)  on  day  1  or  2  of 
the  hospitalization  is  applicable  only  to  the  26  percent  of  patients  who  are  moderately  or  very 
sick  (i.e.,  ASSC  ;>  5)  on  day  1  or  2. 

The  fourth  column  shows  the  percentage  of  patients  with  applicable  criterion  who  actu- 
ally met  the  criterion  for  the  pre-  and  post-PPS  sample  combined.  For  example,  81  percent  of 
patients  hospitalized  with  congestive  heart  failure  had  physician  documentation  about  medi- 
cation use  immediately  before  admission.  Of  the  26  percent  of  patients  with  congestive  heart 
failure  who  were  moderately  or  very  sick  on  day  1  or  2  of  hospitalization,  67  percent  met  the 
criterion  for  ordering  a  cardiac  enzyme  study. 

Table  C.ll 

PROCESS   DIFFERENCE    (IN   STANDARD  UNITS)    BETWEEN   DEAD  AND 
ALIVE  PATIENTS,   AND  PROBABILITY  OF  DEATH,    BY  DISEASE 


(Xi 

Disease 

-  Xo)(1) 
sx 

Probability  of 
Death  within  30 
Days  of  Admission 
(P) 

Acute  myocardial  infarction 

-.42 

.24 

Cerebrovascular  accident 

-.  14 

.21 

Congestive  heart  failure 

-.43 

.  14 

Hip  fracture 

+  .08 

.05 

Pneumonia 

-.18 

.  16 

(1)  Process  difference  (in  standard  units)  between  patients 


discharged  dead  and  alive  30  days  postadmission . 
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PERFORMANCE  ON  PROCESS  CRITERIA  FOR  PRE-  AND  POST-PPS 
SAMPLE  COMBINED 

Tables  C.17  through  C.31  show  the  fraction  of  patients  who  meet  specific  process  cri- 
teria across  each  of  the  five  study  diseases  for  the  pre-  and  post-PPS  sample  combined. 

PERFORMANCE  ON  PROCESS  CRITERIA  FOR  PRE-  VERSUS 
POST-PPS  SAMPLE 

Tables  C.32  through  C.49  show  the  fraction  of  patients  who  meet  specific  process  cri- 
teria across  each  of  the  five  study  diseases  for  the  pre-PPS  versus  the  post-PPS  sample. 

DEVELOPING  PROCESS  SCALES 

In  using  the  medical  record  as  a  data  base  for  measuring  quality,  we  cannot  measure  all 
aspects  of  quality.  Instead,  we  select  a  group  of  processes  that  in  combination  are  a  proxy  for 
actual  processes  delivered  to  patients.  When  process  measures  are  aggregated  from  similar 
measurements  of  the  same  concept,  it  is  appropriate  to  develop  a  scale.  This  subsection 
introduces  the  subject  of  process  scale  development.  In  all  instances,  the  process  measures 
described  here  refer  to  those  listed  above. 

For  this  study,  we  have  several  measures  that,  when  aggregated,  measure  similar  con- 
cepts. Examples  include  initial  MD  cognitive  assessment,  initial  RN  assessment,  and  initial 
laboratory  assessment.  With  the  process  criteria  for  MD  cognitive  assessment,  we  are 
attempting  to  study  a  sample  of  cases  to  determine  how  well  the  physician  evaluates  the 
patient's  past  history  (e.g.,  prior  surgeries),  present  illness  (e.g.,  sputum  or  cough),  and  phys- 
ical examination  (e.g.,  pupils  examined).  We  employ  these  items  as  a  proxy  for  how  well  the 
physician  uses  the  history  and  physical  to  collect  data  about  the  patient.  Our  disease-specific 
panels  believed  that  each  of  these  items  should  be  documented  in  the  medical  record  of  hospi- 
talized patients.  The  idea  is  not  that  a  physician  must  document  each  of  these  factors  in  the 
medical  record.  Rather,  the  idea  is  that  the  more  thoroughly  the  physician  documents  these 
disease-specific  items  during  the  initial  hospital  stay,  the  more  likely  the  physician  is  to  be 
competent  as  a  collector  of  the  patient's  data. 

With  the  process  criteria  for  initial  RN  assessment,  we  are  attempting  to  study  a  sam- 
ple of  nursing  activities.  How  well  does  the  nurse  perform  cognitive  services  (e.g.,  assess 
shortness  of  breath,  ambulatory  status)?  How  adequately  does  the  nurse  perform  technical 
nursing  activities  (e.g.,  blood  pressure  measurement,  weighing  patients  with  congestive 
heart  failure)?  Again  the  goal  of  the  process  items  is  to  sample  nursing  activities  that  aggre- 
gately are  a  measure  of  nursing  care. 

For  initial  laboratory  process  criteria,  specific  laboratory  studies  are  identified  as  impor- 
tant for  study  at  admission  for  hospitalized  patients.  Again,  the  idea  is  not  that  a  physician 
remiss  in  ordering  a  specified  laboratory  study  is  necessarily  a  poor  physician.  Rather  the 
idea  is  that  the  laboratory  studies  we  assess  are  likely  to  be  a  proxy  for  the  physician's  initial 
use  of  laboratory  tests. 

For  each  of  these  three  process  concepts,  the  criteria  belong  together  on  a  scale  if  physi- 
cians and  nurses  who  perform  well  on  one  of  the  criteria  are  likely  to  do  well  on  all,  and  if  a 
negligent  physician  or  nurse  is  likely  to  perform  poorly  on  all  measures  for  that  scale. 


258 


C/5 

05 

< 

05 

Ld 

C/5 

c/) 

Q 

> 

CD 

CO 

4-> 

CO 

Cl 

UJ 

Ct- 

1 — 

Q 

cc 

O  ' — » 

C/5  0 

C/5  C 

0 

UJ  — 

O  JO 

O  E 

CO  4^ 

0J  L_ 

Q-  O 

  Q 

Q  CO 

aa. 

a 

u 

Cl. 

< 

UJ  1 

0 

>  p 

—  </) 

TO 

JO 

1-  o 

Q5  00 

TO 

—  a 

C  P 

h- 

z 

Ol  C 

O  TO 

O  C 

V)  O 

CJ  CO 

00  Q. 

< 

_)  1 

<  0 

—  i- 

1-  Q- 

z 

z 

o 

UJ 

o 

z 

< 

cc 

o 

u. 

a: 

uj 

a. 

C_> 

00 
OO 
05 
O 
O 

i. 

Cl. 

c 


CNJ 
NO 

—  CNJ 
X  II 
C 


J" 

c\j 

<  00 

>  CNJ 
O  II 

c 


o 
J- 

LJ  l~- 
Z  CNJ 

a-  ii 
c 


IfN 
—  CO 

s:  cnj 
<  ii 
c 


•a- 

CNJ 
Ll.  CO 
X  CNJ 
O  II 

c 


■>  o 

00 

co  O 
TO  CO 
05 

co  -P 

P  C 

—  05 
Cl  E 
CO  3 

o  o 
X  o 

TO 

CNJ 

T3 
■P  — 
00  C 
i-  O 

—  X 
0*.  (/> 


0  CO  > 

ceo 

■P  CO  — 

coo 
 i_ 

x  c/5 

P  >>  0 
—  XX 
2  Q.-P 


I  NO  I 

■  r-  i 


I  COM  I 

I  On  CO  I 


I  i-  co  I 
I  I 


I    ON  I 


i  o  co  i 

I  On  On  I 


I  CO  I 
I  On  I 


■3'  I 

NO  I 


I  CNJ  I 
I  CO  I 


I  CNJ  I 
I  0\  I 


I   CO  I 

i  h-  i 


I  r-  I  co  r-  |  \o  Is"  I 
I  UN  I  GOOD  I  COVO  I 


O  I 

NO  I 


I  O  I 

I    NO  I 


I  CO  I  ro 

i  r-  i  r-- 


I  CNJ  O  1  00  fO  I  UN  I 

i  j-  in  i  r-  no  i  -3'  i 


CO  I 

00  I 


I  On  I 


I  co  m  \o  i  no  i 


i  J-  i 

I   NO  I 


I  ' —  lA  I 
I  00  in  I 


iTN  I 
J-  I 


i  in  i 

I    NO  I 


i  r-  i 

I   NO  I 


I   NO  I 


3"  I 
00  I 


I  CO  I 
I    NO  I 


i  on  o  o  j-  in 

I  ON  CO  O  O  r- 


I   CNJ   I   O  00  00 

i  r-  i  r~  cm  owe 


i  onj-  i  r—  i 

I  I*-  00  I  On  I 


I  O  ON  I  CO  I 

i  Is-  r»-  i  on  i 


I  O  I 
I   CO  I 


I   ON  I 


i  i —  r-~    r—  i 

I  On  NO  On  NO  I 


I  CO  3"  I 
I  0N\O  I 


TO 
05 
P 
O 

c 

c 
o 

1/5 

c 

05  TO 
P  0 
T5  S-  -P 
05  05  O 
P  Q.  C 

O  >5 

C  X  0 

1/5 

>><-  CO 
'  O  05 
c/5 
>>  — 
C/5  S-  XJ 
—  O 
X  P  > 

oo  i- 


05 
P 

o 
c 

>> 


o 
p 


o  — 
CO  X 


to  o 

s-  (0 
CO  •- 
O  TO 


"D 

05 
■P 

o 
c 

00 
E 
O 


C5.TO 

-o  E  cu  a 
V  >>*>- 

P  c/5  o  X 

O  C  TO 

C  C(_  U-  (D 

O  X  O  P 


TO 

05 

p 
to  o 

T)  4J  C 

0  P 

P  O  00 
O  C  P 
c  — 
to  X) 
>j-P  CO 
1-  C  X 
O  0 
P  E  Ol 
oo  0  C 
—  Ol— 

x  c  jx: 

co  O  "O 


00   CO  00 

C  O  Z, 

O  O 

._  c_  ._ 

>  o  > 

0—0 

o_  s_  s_ 

Cl.  Cl.  a. 


00 

0  p 

—  Q5 

1-  00 

0  c 

01  o 
s_ 

3  c*. 

oo  o 

p  0 
oo  E 
CO  — 
Cl  i— 


O 

C  00  C 
O  0 

O  O  00 

c  c 

s-  co  O 

OP  — 

oo  j-> 

E  E  co 

3  C-  O 

P  O  — 

c  s-  -o 

Q. —  0 

C/5  O  SI 


"D  —  00 
0  —  CO 
P  E  t- 
O  co  Oi  O 
c 

—  E 

oo  i-  >  oo 
cu  o  


Ol—  o  o 
!-  CO       X  X 

0  —  o  o  o 

—  o  —  o  o 

-  O  i  

<  C/5  OX  <  < 


T3 

0 

P  "O 
O  0 
C  P 

o 

0  c 

00 

3  < 
> 
o  o 
o 

O  S- 
cc  o 

X  — 
O  i- 

l-  a. 


T3 

0 
p  0 
O  > 

c 

00  CO 


•o 

■o 

0 

■a 

0 

p 

T3 

0 

p 

o 

0 

p 

O  T3 

c 

P 

o 

C  0 

o 

c 

p 

00 

c 

oo  o 

P 

00 

c  c 

00 

s_ 

■O  T3 

3 

0  co 
a  p 
E  c 

0  0 

r-  Z 


Cl  Ol 
ct  ^ 


X  1- 

0  x 

C-  T3 

IU  — 
1-  P 
O 

01  i- 

CO  X 

O  O 


C  E 


"O  0 
0  P 
P  o 
o  c 
c 

p 

O  co  co  co 
0  0  0  0 
Z  X  X  X 


_  1_ 
o  r; 

oo  E 


TO 

0 

TO  P 

0  o 
P  c 
o 

c  c 

0 

00  E 

01  o 

C  T3 

Z.  X 

-1  < 


■o 

0 

p  -o 

0  0 
C  P 

TO  O 

X  0  c 

■p  p 

01  O  CO 
C  C  E 

0      0  -o 

S-  0  TO  0 
P  00  0  P 

00  —  O 

;>>c 
cap 

O  —  oo 

Ol  E  0 

0  0  0  X 

C  —  i-  0 

O  P  — 

■P  S-  X 

O  0  0 

0  s- 

  5_ 

O  CO  0  X 

oo  "O  >  E 

-j  05  O  — 

s:  a.  _ i  _i 


X  X 

i-  CM 
Q  Q 

•O  "O 

0  0 
00  00 
00  00 

0  0 

00  00 

oo  oo 
co  co 

CO  CO 

o  o 

CO  c/5 


X  X 

r-  CNJ 
II  DDI 
i-  CNJ  •  •  r- 
Q  Q  TO  TO  Q 
..00. 
05  0  P  P  TO 
CCO 

0  O  C 
T3  TO 

c 

EE  

co  co  co  co  oo 
X  X  co.  a  p 

05  0  — 
JJ  P  o 

01  ai  oo  oo  — 
C  C  0  0  u- 
LO  Zj  X  X  0 

X  _l  o  o  o 


o  0 

C  P 

o 
c  c 


X 

CNJ 
Q 

TO 
0 
P 

o  X  X 

C  t—  CNJ 

a  q 

in  •  • 
p  0  0 

—  +J 

o  o  o 

—  c  c 
IP 

0  Q  Q 

Q  2  Z 


CO 

s_ 

0 

p 

0 

!_ 

X 

O 

CO 

o 

00 

00 

0 

a 

O 

a 

O 

CO 

s_ 

a. 

0 

s_ 

Cf- 

CO 

O 

00 

s. 

p 

0 

c 

X 

0 

E 

d 

p 

z 

CO 

Q. 

CO 

■p 

o 

< 

1— 

259 


00 


o 

0 


-O 

TO 


CO 
< 
bJ 
CO 

Q 

> 
CD 


CC  -D 
Ld  0 
I-  C 

ce  xi 

O  E 
O 

CO  O 
CO 

uj  co 
O  a. 

O  0- 

cc  i 
a.  4-> 

10 

z  o 
a 


en 


— I  -D 

<  C 

—  to 
h- 

—  I 

z  0 

—  s_ 


UJ 

o 
z 

< 
z: 
cr 
o 
u> 
on 

Ld 
C 


CD 

C\J 

Cu  \jo 

1_ 

—  ^ 

CD 

X  C\J 

4_) 

II 

4_ 

o 



to 

<t  CO 

oo 

>  CO 

0 

CJ  CM 

O 

II 

o 

c 

C_ 

Ct_ 

Ov 

C 

z  r~ 

a.  cm 

4-> 

ii 

0} 

c 

CD 

ro 

to 

—  iA 

4-> 

2:  co 

c 

<  C\J 

CD 

n 

c 

4-> 

CO 

0. 

J- 

Lw  CM 

t»- 

X  00 

o 

O  CM 

&s 

II 

c 

X)  to 
TO  4-> 

a  c 

—  o 

a  a. 

a 

< 


"D 

CD  CO 
C  4J 
Ol  C 

co  o 
CO  Q- 

< 


CD 

4-> 
4- 

o 

to 
to 
CD 

o 
o 

L. 
Ct- 


4-> 

—  c 

TO  CD 

4J  E 

a  o 
to  o  •• 

OT3  Ol 

x:  c 

TO  — 

CM  -  > 

3  O 

4->  O  — 

to  JZ  — 

4-  to  o 

—  o- 
Ct-  to 

CD  CD 
CD  CO  £1 

r.  i-  -p 

C  <*- 

c  o 
x:  to  x: 

■P  >>  O 

—  TO  TO 
3  "D  CD 


I  I  I  I 
I    I    I  I 


I  I  I  I  I  I 
I    I    I    I    I  I 


I  I  I  I  I  I  ro  CO  CM  On  I  ro 
I    I    I    I    I    I  lAJ-vOCO  I  CO 


rOOOIIIIIIIIIJj 

t*»  Is-  i  i  i  i  i  i  i  i  i  r>- 


I  O  CO  CO 
I  \0C0On 


i  j-  r-  r~ 
i  vo  r>-  o 


I  i—  ro  t~~ 


O  O  I    I    I    I    I  I 

ON  G\   I     I     I     I     I  I 


X  X 

r-  CM  -» 
Q  Q  CM 


r-  CM 
q  a 

TO  TO 

CD  CD 
to  to 

to  to 

CD  0 

(0  (0 

(O  (O 

TO  TO 

co  co 
o  o 

co  to 


X  X 

»-  CM 
Q  Q 

■a  to 

0  CD 

to  to  X 

to  to  i— 

CD  CD  Q 

to  co  • 

tO  tO  T3 


TO  TO 
C  C 


CP 
oo 

to 

CD 

TO  TO  CO 

a  a  oo 

TO 

4->  4-> 

to  to  to 
CD  CD  O 

x:  x:  > 

CJ  O  Q- 


X  Q 

CM  • 

a  -o 

•  CD 

TO  (0 

CD  00 

00  CD 

00  to 

CD  00 

00  TO 
00 

TO  00 


"D 

CD 
.p 

o 

C 

X 

CM  CD 
Q  -X 
•  TO 
■D  -P 
CD  C 
00  — 
00 

CD  — 
00  TO 
00  c 
TO  O 

00  4J 


00  5. 

o  a  a  p 

O.CLQ.Z 


TO 

CD  X 
P  CM 
O  Q 
C  C 

i-  a> 

d)  O  -C 

to 

TO  X  4-3  X 

P  r-  CM 

CO  (BO 

—  .  s_  . 
C  3  00 

—  CD  00  CD 
TO  -X  00  £ 
C  TO  CD  — 
04JLP 

a 

P  4J  ro 
•-  XT  TO 
s-  Ol  O  II 
u-  OA 
3  CD  — 
Z  ^  CO 


TO 

CD 

4-! 

o 

73  C 
X  0  TO 
r-  P  CD  CD 
O  On 

•  c  o 

ro  C 
CD 

II  O  oo 

A  C  3 

CD  4-> 

to  c  TO 

C_ —  P 

—  4->  00 
4-  C 
4->  O  >5 

1/1  O  L 

o 

!->>«- 
O  i-  TO  E 
p  to  —  to 

—  C  3  TO 
C  —  XJ  CD 

b  s-  E  s- 
s:  =>  <  a. 


o 
c 

CD 

■a 

00 

CD 
4_ 

c 
o 

10 
00 


00 


I  I  CM  CM  iTi  ro  I  I  I  I  CM  I  r-  I  I  ro  r— 
I    IOsOvOI^I    I    I    I  NO  I       001    I  ON 


I    I  r-        I  OMAJ  VO 

i  i  oo     i  \o  os 


CD 
4-> 


00 
00 
CD 
O 
O 
s_ 
a 

Ci- 
O 


TO 
4J 

o 


O 

0) 

CO 

t/5 

T3  C 

0 

c  o 

CO 

CD  — 

CD  CO 

1_  (+_ 

0)  4J 

I—  c 

4->  CO 

f~  0 

4_)  > 

>  4_) 
->  4-< 

oo 

TO  CD 

•—  o*i 

t_  c 

CD  O 

\j 

C  4- 

o  o 

to  J£ 

oo  o 

CD  O 

o  x: 

O  00 

a  4J 

3 

T3  O 

<D  C 

4->  4J 

to 

00  — 

4J 

—  X  c 

0 

CD 

£  oo 

4J 

cj  o 

TO 

to  x: 

Q. 

CD  4-! 

d_ 

4.  CD 

o 

O  s- 

t>-  TO 

4-> 

c 

CD  — 

0) 

-  -z. 

o 

X3  < 

4. 

TO 

0 

O  <- 

a 

—  o 

—  (t- 

a 

Q.  00 

TO  4-) 

C 

0  CD 

0 

1_  — 

TO  4-> 

TO 

00  CL 

4-J 

CM 

C  0 

0  — 

TO 

—  X! 

c 

4->  TO 

CO 

TO  O 

a— 

-  o-  >> 

—  Q.  TO 

<  <  TO 

r-  CM 


260 


c/5 

< 

UJ 

in 

— 

> 

CD 

c/5 

</> 

u 

cj 

o 

tr 

a. 

2: 

< 

o 

o 

TO 

0 

a 

C 

< 

JD 

cj 

E 

o 

O 

cr 

o 

i— 

o 

UJ 

0. 

_J 

a. 

o 

UJ 

i 

p 

0 

Q 

l/> 

z 

O 

JD 

< 

a 

CO 

1— 

> 

TO 

< 

c 

cr 

CD 

i 

X 

1 

0 

y- 

!_ 

<r> 

UJ 

X 

o 

_l 

< 

r- 

2 

o 


o 

< 

s: 
cr 
o 

u. 

cr 

UJ 
Q. 


ro 

</> 

CO 

4J 

ON 

C 

c 

c_ 

0 

CD 

p 

+J 

+j 

CD 

CO 

a) 

S_ 

a. 

O 

JO 

CO 

o 


—  o 
aa. 
a 
< 

TO 

CD  O 
C  4J 
CTI  C 

i/i  O 

yi  a. 
< 


CO 

t/i  — 

Q)  0 

O  *-> 

o  — 

s.  s_ 

Q.  O 


P  — 

c 

CD  4-> 

—  CD 
p  u) 
cc  -Q 
0-  3 

C/5 


0 

co 

CO 

0 
Cfl 


in  on 
ON  oo 


cm  m  o 

ON  On  On 


On 
ON 


NO 


O  J-  co  CM 
00  00      ON  ON 


CM 

0  — 

.£3  l/> 

CO  4J 

o  c 

o  o 

—  CD 

o  o 

au 

o.  to 

<  0- 

o  o  o 
o  o  o 


o  o 
o  o 


o  o 
o  o 


C\J  C\J 

co  co  co 

CM  CM 

CM  CM 

CM  CM 

CO  CM 

CO  CO  CO 

CM  CM 

CM  CM 

CM  CM 

>> 

>> 

>> 

>» 

to 

co 

CO 

to 

CO 

(_ 

S_ 

s_ 

S- 

t_ 

1 

X 

X 

1 

X 

1 

X 

X 

4J 

•P 

4-> 

p 

4-> 

UJ 

U5 

t/5 

in 

CD  O 

flJOO 

cd  a 

0  CJ 

0  CJ 

SZ  XL 

JZ  ^  X. 

jz 

c  ^ 

jz  ^ 

CJ  UJ 

CJ  UJ  UJ 

o  uj 

CJ  UJ 

CJ  Ld 

CM  CM 

o  o  cj  o 

CM  CM 

CM 

mi-  ini- 

CM  CM 

CM  CM 

t- 

>>  >>  >>  >> 

s_  !_ 

i_  C_ 

o  o 

O 

co  s-  to 

O  O 

o  o 

CD  13  CD  TO 

i-  i-  CM 

>  > 

>>>>>> 

i-  lf\  S-  lTN 

>>  > 

>>>> 

co  to 

co  co  co 

O  O 

to  co 

CO  to 

TO  TO 

TO  TO  TO 

II  II 

TO  TO 

TO  TO 

>>A  >>A 

U5  (/) 

U)  (/)  C/> 

W  C/5 

yi  co 

■P  -P 

p  p  p 

0  O  CD  CJ 

4->  P 

4-1  4-> 

c  c 

c  c  c 

P  C/5  4->  C/5 

c  c 

c  c 

CD  CD 

cd  a)  cd 

CO  t/0  CO  C/5 

0  0 

0  0 

!_<,!_< 

4->  P 

4->  P  P 

0)  0 

4->  4-> 

•P  4-> 

co  to 

CO  CO  CO 

TO    «T0  « 

CO  CO 

to  co 

a  a 

a.  a  a 

O    •  O  • 

a  a 

a  a 

e  0  e  0 

t^ ._  ._ 

<  < 

<  <  < 

<  < 

<  < 

a 


UJ 

a. 


< 

> 

CJ 


0 

o 

U5 

sz 

0 

4-> 

0 

</> 

c 

CO 

o 

0 

•~ 

t/I 

10 

u) 

TO 

E 

JZ 

"D 

CJ 

CO 

CO 

0 

0 

£. 

JZ 

O 

0- 

0 

JD 

1/5 

J£ 

4J 

0 

C 

0 

0 
•~ 

> 

4-> 

0 

co 

JZ 

a 

p 

4- 

c 

O 

c 

a 

o 

UJ 

— 
4-> 

•  o 

"O 

0  CO 

c 

—  L. 

•  to 

co 

co 

—  >> 

CJ  CD 

!-  CO 

—  sz 

0  s- 

—  4-> 

P  1 

a 

—  X  CM 

Q.  (/) 

s_ 

CO  — 

O  4J 

O 

0  0 

</)  0 

!  

U5  JZ 

CO  J2 

0  O 

to 

0 

>> 

CO  CJ 

O  CO 

to 

i_ 

TO 

s  

Q.T3 

0  a 

to 

c 

•u  a 

0  JZ 

o 

—  CO 

JZ 

s_ 

P  0 

1/1 

CJ  0 

i/i 

> 

p 

!_ 

P 

p  co 

c 

t/3  CO 

0 

0  JZ 

0 

If) 

0 

CO  tCJD 

E  -O 

4-> 

o  — 

CO 

O  1- 

i/l 

O  3 

a 

1_  cd 

jz  o 

Q.4-) 

4-> 

>  o 

s_ 

C 

0 

E  S- 

0 

1/1  t/I 

o  cj 

p  p 

u 

JZ 

4-> 

c  c 

>  E 

(O 

0  0 

V) 

O 

a 

o  sz 

p  p 

0 

co  co  JZ 

o 

a  Q.4J 

V)  O 

4->  4-> 

c 

0  CO 

>> 

c 

o 

CD  tf> 

J3  C 

c 

—  4-> 

CO  O 

o 

•P  c 

0  E 

CO  0 

c 

-3d- 

a.— 

—  0 

o 

P  Ct- 

a.  c 

U)  CO 

0 

a  Q.T3 

0  Q.T3 

CO 

0 

c 

0 

s_ 

—  Ci_  T3  C|_  O 

tt-  O 

0 

O  C 

3 

0 

p 

CT 

Q  &S  U5  &5C/0 

0 

S_ 

>-  CM 

CO 

0 

0 

{_ 

jr 

0 

p 

261 


o 


re 


Cu 

i  \o  r~  o  i-  vo  Is-  i 

— 

1  NO  00  On  On  On  ON  1 

X 

to 

LU 

re 

— 

Q 

— 

03 

< 

CM  r-  00  i-  i-  in  J-  1 

1— 

4-> 

> 

00  Is-  00  On  On  On  On  1 

to 

— 

o 

> 

O 

o 

o 

Ol 

_l 

c 

o 

— 

LU 

OCNjr-OOooaOiTN 

1— 

4-1 

z 

oor^ooo\ONONO\ON 

< 

0) 

a. 

s: 

0) 

lu 

z 

X 

CO 

a 

*J 

z 

c 

J-  CNJ  ON  J?  J"  LfN  NO  1 

< 

0) 

00  f*"~  CO  On  On  On  On  1 

< 

> 

4J 

a: 

re 

t- 

Cu 

to 

— 

<<_ 

o 

Ll. 

ro  lA  O  tf*\  lA  f>  ro  1 

lu 

X 

CO  r~-  ON  On  On  On  On  1 

X 

O 

o 

a 

- — - 

o 

u 

o 

CO 

_J 

C 

co 

— 

—  CO 

4-1 

cu 

E 

re  c 

o 

O 

O  CO 

OOOOOOOO 

o 

—  — 

OOOOOOOO 

—  4-1 

< 

to 

q.  re 

— 

o_ 

O-Cu 

cc 

LU 

a. 

< 

h- 

i 

4-1 

— 

CO 

O 

O 

a 

— 

X3  CO 

o 

"O 

re  4-1 

z 

c 

O  C 

— 

re 

—  •— 

T—  1—  1—            T—   1—T—  f— 

a: 

—  o 

LU 

i 

an 

a 

a) 

CL 

cc 

s_ 

< 

O 

a. 

> 

tr 

O 

0)  co 

C  4-1 

< 

ai  c 

tr 

—  — 

O 

ifi  O 

CD 

i/)  a. 

< 

< 

• 

CO 

—I 

— 

n 

< 

re 

— 

L. 

o 

1- 

0) 

— 

— 

—  -o 

— 

Z 

re  s- 

a 

— 

■u  O 

a 

— 

re 

z 

Q.-D 

o 

co  — 

•  • 

CO 

O  3 

cn 

LU 

— 

£  O 

c 

re 

o 

i — 

.C 

z 

— - 

C\J  CO 

0) 

CO 

< 

o 

4->                          4J  4J 

4-> 

21 

re 

4-1  CO 

re               c  c 

C 

a: 

— 

CO  c 

C                    3  3 

CO 

o 

s-  re 

o 

o             o  o 

— 

li- 

CO 

.0              o  o 

re 

ce: 

*J 

<»-  o 

i_ 

re 

LU 

C0 

re              -o  tj 

P 

Q. 

Q. 

L. 

CO  CO  .c 

O  CO               O  4->  O 

c 

O 

-C  >>4-> 

—  C           E  O  —  O 

a> 

4->  .C 

n—  3—i  

!_ 

co 

C            —  -O  O  -O 

CO 

< 

co 

c 

O 

co  —      E  co  o 

Ct- 

a> 

34->         3   CO   CO  4->  CO  1>- 

o 

x:  co  jc 

O  re     —  re  4->  re  ■»-> 

o 

4->  >,  o 

C  DZ-OU-  E  — 

!_ 

—  re 

re 

J)  L3  o  or  J)£ 

Cu 

0) 

>ocomcl  jij 

262 


u 

0 

n 

CO 


CO 
< 
UJ 


> 

CD 

CO 
< 

o 

Q 
O 
O 
_l 
CO 

_J 
< 

cc 

UJ 
h- 

az 
< 

Q 
z 

< 

Q. 

O 

or 
o 


< 

CC 


cr 

o 


Q 
CC 
O 

> 

ce 
o 
i- 
< 
cc 
o 

CD 
< 


O 

z 

< 
s: 

o: 
o 

u. 

cc 

UJ 
0. 


oo  en 

P  C  CD 
C  

CP  4.)  i- 

—  CD  0 
p  (D  4-> 

cc  s:  — 
a.  s- 
o 


0  c\j 

S3 

co  oo 

O  4-> 

—  C 

—  0 
a.-- 

Q.  4-> 

<  cc 
Cl 


£3 
CO 

u 


—  o 
an 
a 
< 


TO 

0)  00 

C  4-> 
Ol  C 

V)  O 

GO  Q- 
< 


CD 
1/5 
« 
0 
oo 


p- 


00 


CM 


C\J 


-3- 

.3- 


m 
J- 


3- 

in 


ro 


m 


C\J 


o 
o 


o 
o 


o 
o 


o 
o 


■3" 
ro 


o 

ro 


C\J 


CNJ 


00 


0J 


00 


00 


00 


00 


00 


X 

o 


UJ 

z 

Cl 


oo 
m 


oo 


oo 


oo 


X 

00 

X 

oo 

OO 

Q 

CD 

it 

o 

0 

CO 

_l 

E 

a. 

>> 

_) 

E 

CL 

>> 

00  — 

>> 

>> 

> 

o 

CO 

>> 

O 

CO 

>> 

>> 

>> 

>> 

I/)  s- 

CO 

CO 

(_ 

N 

T3 

N 

T3 

CO 

co 

co 

CO 

0  CD 

TJ 

TJ 

O 

C  i- 

o 

C  00 

■a 

T5 

TJ 

T5 

O  4J 

00 

oo 

CD 

co 

>5 

CD 

co 

> 

oo 

oo 

oo 

OO 

o  — 

o 

X. 

O  >> 

P 

X. 

o  >> 

4-> 

o 

o 

o 

CD  i- 

Cl. 

00  CO 

o 

c 

a. 

OO  CO 

o 

C 

CD  c- 

Cl.  S- 

CD  t- 

CO  s- 

Q_  O 

o  o 

<  O 

CJ 

—  T3 

r- 

o 

O 

—  T3 

I— 

o 

<  O 

O  O 

<  O 

<  o 

Ji 

L£ 

L£ 

o 

o 

O 

O 

CD 

o 

10 

OO 

OO 

00 

oo 

oo 

>> 

>> 

r— 

00 

>> 

>) 

>> 

>> 

i_  — 

s_  . — . 

s_ 

1-  - — 

c  • — ■ 

c_ 

c  m 

cd  m 

>> 

>> 

■a;  in 

a  in 

cd  in 

CD  OO 

> 

> 

CO 

CO 

> 

> 

> 

> 

n 

ii 

"D 

T3 

n 

n 

ii 

Ii 

A 

1-  A 

<-  A 

t-  A 

A 

5-  A 

o 

O 

OO 

oo 

O 

O 

o 

O 

o 

o 

4-> 

4J 

o 

o 

o 

O 

>>co  oo 

>>co  oo 

c 

c 

>co  oo 

>>co  00 

>,co  oo 

>CO  00 

—  CO 

—  co 

CD 

CD 

—  CO 

—  CO 

—  CO 

—  CO 

4->  -~- 

CD  <  S- 

CD  <  S- 

CD  <  S- 

CD  <  S- 

CD  <  S- 

CD  <  s- 

c 

P  O 

4->  O 

4-> 

4-> 

4->  O 

•L>  O 

•P  O 

■P  O 

CD  4-> 

CO  ■> 

CO  •> 

CO 

CD 

co  » 

CO  •> 

CO  - 

CO  - 

—  0) 

5-     .  i— 

S-    .  I— 

a 

a 

S-     .  i— 

1-    .  1— 

1-     .  r- 

4->  O0 

CD  CD 

CD  CD 

CD  CD 

CD  CD 

CD  0) 

0)  0 

CO  -Q 

■a  •  >> 

TO    ■  >) 

•o    •  >> 

■o    •  >> 

T3    •  >> 

TO      •  >5 

Cl.  3 

o  —  CO 

O  —  co 

O  —  CO 

O  —  CO 

O  —  CO 

O  —  CO 

CO 

S:  -— T3 

s:  —-a 

< 

< 

£  — Q 

s:  —a 

s:  •— q 

2  ~Q 

< 
> 


4-> 

o 

JZ 

■ 

UJ 

z 

CD 

Cl 

oo 

CO 

T3 

CD 

C 

00 

CO 

TO 

SZ 

2. 

o 

< 

CO 

CD 

u. 

.C 

X 

4-> 

O 

L. 

O 

oo 

c*- 

4J 

c 

T3 

CD 

0 

'— 

00 

4-> 

CC 

d 

oo 

4^ 

c*- 

C 

O 

0 

c 

C 

o 

o 

a. 

E 

4-3 

O 

•  o 

o 

CD  CO 

—  1— 

• 

^3  C»- 

CO 

CO 

—  CD 

O  CD 

SZ 

—  SZ 

CD  4-> 

—  4-> 

4-3 

ct 

—  c 

Q.  00 

^  co 

CO  — 

CJ  SZ 

4-> 

CD  CD 

oo 

I  

oo  i- 

co  -Q 

0  0 

CO 

O  ^1 

CO  CD 

O  4-> 

i-  CO 

i  

a  s- 

cd  a 

4->  Cl 

0  oo 

—  co 

^  4-> 

L. 

4->  C 

a  0 

OO 

0 

p 

4->  C 

00  CO 

0 

0  O 

oo 

OO 

0  a 

(1)  ro  J3 

E  E 

o  — 

Zj 

O 

O  i- 

00 

o  o 

S.  CD 

SZ 

au4J  jin 

c 

E  s- 

0 

00  c 

O  O 

4-1  O 

4-> 

zz 

y  e 

CC 

0  TO 

o 

a 

—  0 

O  SI 

4J  05 

4->  >  d- 

CO  CO 

o 

00  O 

4J  4-> 

c 

0  oo 

c 

o 

0)  oo 

£} 

—  4->  4->  CO  < 

•P  C 

O  > 

CO  0 

c 

—  o 

a-- 

4J  U. 

a  s- 

OO  CO 

0 

a  o 

CD  Q.T3 

co  <*- 

c 

—  C<_  TJ  c>-  O 

o 

0 

o  co 

0 

p 

co 

<-  00 

CO  J- 

0 

4-> 


263 


CM 
CM 


< 

or 


or 
o 


o 
o 


>  — 
h-  it 

UJ  c 

z  — 

U  £> 

-j  e 

UJ  o 


O  to 
Z  0. 

<  o. 
I 

3  « 

O  w 

—  o 
u. 

OT3 


<  — 


o 

z 

< 


or 

o 


ro 

V>  ro 

*-i  c  — 

C  —  i. 

-—  ©  £ 

o.  o 

< 

CSJ 

> 

©   

u 

re  ■*-» 

o  c 

—  © 

a  «-> 

a.  co 

<  o- 

tfl 

*j  a*  co 

c  c  — 

rj>-  t_ 

—  u  O 

~  c 

co  ©  — 

a-  Z  <- 

O 

UJ 

CM 

z 

CD  _ 

a. 

J3  tfl 

ro  *-> 

o  c 

—  © 

a.  ro 

<  o- 

ro 

*->  C  ro 

c  

0)  *J  L. 

—  OJ  0) 

*j  a> 
to  z  — 

0-  L. 

o 


a  — 
a  *-> 
<  <o 


</)  0"i  TO 
*J  C  — 
C  —  t- 

aj  4->  © 

—   0)  4-> 

to  Z  t. 
a.  u 


(0  4J 


Q.  TO 
<  CL 


V 

TO  *J 
O  C 

-  O 

a 
< 


w  0 

a. 

< 


O  On  CO  O  O  O  I 
C\J  W      *-  f-  t-  I 


I      I      I      I      I      t    OsJ  J 

I    I    l    I    I    I  Ov 


Or-  CM  C\J  O  CM 

co     co  »—  ao  *— 


vO  ^  ^  CO  30 


i  -3-  \o  r-  in  i  i 


I  r-  r-  II 


CMCMroror-r-r-r- 


CNjf-focNjr-por-f 


r—  r—  >o  vO  *o  xO 


r~  r-  \Q  <M  VO  CM 


J-      J      ro  |  r-  I 


■3  cm  j  m  m  i  r-  i 


>>  >> 

>>  >. 

>>    >  >> 

>1 

>>  >> 

>. 

>> 

>>  >> 

(0 

t_  I, 

L>  t_ 

t_       t.-  L. 

L. 

L 

s_ 

i_  C_ 

Ul  — 

4-1  *J 

4J 

4->  4J 

4J 

*J  4-> 

*j 

+J  J-> 

01  t- 

tu  a, 

oi  a) 

o)     0)  a) 

S 

«J  CD 

CD 

CD 

CD  CD 

©  © 

E  £ 

E  E 

SEE 

e 

£  E 

E 

E 

E  £ 

O  *-> 

©  © 

IS  « 

a>    a>  o 

o 

01  CD 

CD 

o 

CD  CD 

o  — 

3  —  3  — 

—  3  — 

3  —  3  —  3  — 

3  — 

3  —  3  — 

3 

—  3 

3  —  3  — 

(.  l_ 

u  ©  o  a)  o 

OIU  I 

(J  d)  <J  ttJ  O  CS 

o  <s 

U  CD  U  CD 

u 

ID  U 

CD  U  CD  U  CD 

c  o 

OHOKOKUl- 

UHUI-Ot- 

U  1- 

UhUh 

UHUI-Ur-UI- 

rc 

r 

C 

a> 

— JC  —  vo 

*J 

ro  — ~ 

1  1 

r- 

OJ  -o 

1 

Pw      I  E 

u>  r 

1  J 

c*i  m 

C71  C 

—        in  w 

ro  W 

m  o 

CM 

CD 

< 

CM 

j: 

ii 

ro  ro 

II  tfl  II  t. 

U 

11  JJ 

i. 

L. 

JC  II 

II  U  II 

A 

E 

A       A  O  II  CO 

II  CO 

v     ii  a 

II 

O 

O  W- 

O  II  A 

-X  — 

(0  * 

>>     —  0) 

o 

0 

u 

JC  E 

E 

o 

ou  »u 

U 

OI-UKOIO 

uro 

u  o  o  o  > 

U 

E 

«  (J  O 

—  Ul  to 

in  a£ 

CO 

C/5  0)  to      (/}  C 

t/>  to  c 

u)  to  —  tn  —  « 

CO 

o 

O  CD 

(O  C/) 

vi  in  >>co 

t0  ~  4J 

c  ~ 

C/5  >  tO  >it0  QJ  CO 

to  CO  CO 

tO  <A  to  IA 

1/} 

c 

iJ  O 

>.!/)  CO 

<  -  <  JC  <  c  — 

<o  E 

O  <      <  —  <  > 

<  > 

>><       <  4! 

< 

a  c 

—  <  < 

4J  © 

u 

UJ  > 

(-       o      —  0) 

ro 

—  CD 

+>      >.  E 

c 

jr 

£  CD 

CD  U 

4J  V) 

ro   -  *j  « 

■■  CI 

t.  w 

to 

t- 

»  ro    -  i-  O 

>>T5 

*J  * 

C  « 

£    ■  IS  ■ 

01 

•  o  j: 

o  >> 

■  CO    •  trt  3 

> 

.  o  3 

> 

•  JZ    •  CD  (/) 

to  — 

CO    •  ■ 

©  © 

)  ©  t-  © 

0)  —  (j 

r 

>.  ca    o>  <-  eg  oi  — 

0! 

CI)  0)  — 

ID  )  CD  > 

CD 

5 

> 

t-  CD  CD 

—  to 

u  .  o  . 

> 

■"O  OT3 

t. 

.  y    .  0)    ■  CV- 

E 

.  rji— 

.  CD    •  4-> 

CD  CD 

CD    •  >>  • 

*->  J3 

E  -  -o  — 

L. 

—  t-  JZ 

c  t. 

—  O  —  T3  —  C  CO 

O 

—  C  10 

0 

—  E  —  *>  3 

L. 

C 

•O  —  <  

(5  3 

o  —  o  — 

V 

—  ro  ifl 

0)  CO 

> 

—  £  —  O  —  O  <- 

V> 

_  O  c- 

c 

—  o  —  o  a 

O 

z 

o  o 

O  —  CD  — 

CL  CO 

w  z 

> 

o 

u7    r  o 

O 

to  z 

< 

z 

z  > 

13 
V 


CD  CO 

—  c 

a  c» 

CO 

CO 

O  CD 

L. 

—  £ 

©  to 

—  4-> 

4^  AJ 

a. 

—  c 

a.  tfl 

L.  © 

CO  — 

O  — 

CD  CD 

to  CO 

I  

CO  -O 

© 

CO 

U  © 

CO  o 

O  I- 

1-  3 

I  

a  - 

CD  a 

u 

4->  a 

©  ro 

—  CO 

£  t- 

t- 

*J  w 

O  CD 

L. 

4j  a 

tfl  ro 

©  _ 

to 

©  JZ 

©  CO 

E 

rj  ~ 

L 

O  l- 

O  O 

<-  © 

JZ  t»- 

a  *j 

> 

ro 

E  t- 

ty>  ■— 

o  o 

4-1  t. 

C  © 

>  £ 

©  4J 

O 

4J  1- 

^  > 

co  o 

a 

to  o 

>> 

*->  *j 

©  c 

c 

©  Ui 

ii 

—  4J 

■t->  c 

u  © 

CO  © 

a— 

ul 

—  « 

*J 

a  *j 

to  co 

a> 

a\ 

©  a.  vi 

CO  3 

c 

JD 

O 

— 1>_ 

3  V.— 

w  o 

VI 

o 

© 

o 

c 

© 

r-  CM 

co  JI 

CO 

ro 
C7i  — 

C  TO 

c 

CJ 

*J  L. 

t/> 

CD  O 

tf 

■P 

CJ  *J 

to  r  — 

CJ 

L. 

o 

<  I> 

>  TO 

CJ  0. 

CJ  0J 

d- 

O 

.O  to 

TO  ^ 

U  C 

—  o> 

a  w 

a  re 

<  0- 

I   I   I   I  I 


I     I     I     I     I  I 


1   1  r-  1  «-  1 


1  1  w  1  j  1 


4-»  c  n 

c  —  — 

DiJL 

Ui 

_  n,  » 

*J 

*J  0)  4-> 

c 

ro  21  — 

V 

0-  u 

0 

UJ  *-> 

z  ra 

a.  a. 

tp  <\j 

b- 

O 

&  tfi 

co  *-> 

V? 

0  C 

—  a) 

au 

a  rs 

<Q. 

ro 

tn  ai~- 

c  ra 

c  

0  **  u 

ul 

4)  1) 

4-1 

^  CD  ^ 

C 

to  E  — 

a> 

a.  u 

O 

—  *j 

r  cd 

<  a. 

V  CM 

O 

CO  4-» 

0  c 

a  to 
<  o. 


1  1  1  1 


1   1   1  1 


ON 


O  co  r—  O  00  r- 
co     co  00 


o  co 


J-  —  r-  iTi 
CO  —  CO  1- 


r-  r-    u*ma  1  1  1 


1  1  1  1 


OOOO 


O 


o 


cp  J->  t- 
V)  —  on) 
i-J  4J  CD  w 
C  (Ol- 
co  a.  l. 
u 


I     I     I    I    I  I 


00.  ~ 


n  V) 
TO  -P 
O  C 


Q.  ■!-> 

a  to 

<  CL 


O  C 


CJ  i/5 

en  c 

V)  O 

< 


C  4-> 

cd  cj 

—  (/> 

TO  3 

a.  tO 


1  1  r-*  r-  <m  cm  1  1 


1  1  1  1  1  1 


1  1  1  1  1  1 


U  DO  (DU  (DO  l) 


r*>  r*-  \£>  vo  ■O  vo 


r-  f-  \0  cm  ^  CM 


CJ      CJ  CJ 

£       E  E 

CJ       CD  CP 

—       —  — 

CJ  CJ  O  CJ  O  CJ 


LP.  CM  J"  CM 


CU  0) 

E  E 

CJ  Q> 

3)  —  3>  — 

u  a;  o  a> 


Jrort  l  I 


CD 


CD 


tD 


3  — 
U  CJ 
O  1- 


E 

CD        Cu  CD 

=3  —  id  —  r>  — 

u  coo  cvu  id 


TO  CD 


u  11 


E  — 
C  — 

II   O  C  *-> 
AUTO 

V)  CM!  o 

o 


V  O  II  TO 


0 


g 


11 


UU  WU  CJ  (-  t-  —  Luwoio  -  UXO-  U  O  CJ  CJ  >> 
■-tO       tO       I/)  OO  V)- ■  CJ  CO       tO       Z  tO  i/l  ZtfttO-tO-Ul 

wi  to  >>to     1/)  ■—     <     r~  >  to  >>t0  cj  <     to<D<     to  m  to  </> 
<  —  <  jc  <  c  -o     >>        <  —  <  >         <  >      >i<     <  QJ 

*J       0)       O       CJ  C    *  l-  i!  C£       CJ      —  CJ  TO      ■-  CJ  L.       *->       >>  E 

(Q      .  4-J  ■    n-  L.    1    01  A   O      "  — 1      .        t-  £      k  *J   t_    Qj      ..  flj      .  l_  o 

.C     •  (U  -tO     •  O        C7»>              -TO     •  i/>  3  >     •  «  3  >     •  .C  .Q)«i 

>  CJ  t.  O        d>       .C  CO       CJ  ^  Cjt-CJCD—  CDtDCJ—  Q)>CJ> 

CJ    -  Q>  .  >»    >T)  D  E  QtO  O    -  CJ     -  0>--  £    ■  0*1      iJ    •  CJ     .  4J 

E      -q       1  (  ro  2  CO  O  —  TJ  —  C  TO  O  —  C  TO  O  —  E  —  *->  3 

O  ■ — ■  O  - — >  CD  • — -as  >0<<^ZwO  —  OU-tO  —  Ot4-Z  —  O— -  O  C 

toz>o  tozo  CJ  toz 


TO  C  TO 

r  aia 

4->   — 

— «  (ft 

(M  cu      to  < 

(_  t_  CD 

oxw 

U  O 

E 

O  C  CD 

+J  TO  O 

ax:  c 
E  *j  V 
>>  -o 
1^  1 — 

CJ  > 
CJ  JZ  UJ 

c  w 
000 


II  o  ■ 
E 

o 

to  o 
to  c 
< 


O  CM  A 

to  to 

>>to  to 

-  <  < 


CU 


CD 

CO    -  >> 


•  0 

CP  TO 


O  CD 

—  £ 

—  *j 

a 

a  lo 


CJ  CJ 

L>  — 
TO  JD 


0)  (A 
*i 
—  C 
C  CD 
CJ  •- 


f)  a 
u 

O  CJ 

TOO    •  O  »- 

 W  (-  3 

t_  —  <J  O.  U 

cj  a  cj  cj 

■M  a  «5  CD  TO 

—  1- 

U  CJ  to 

**  a 

(fl  TO        CJ  — 

w  cor 

CD  TO  CD  E 

y   t_ 

O  1-  O  O 
<-  CD  TO  .C  <^ 

awa) 

(0 

O  O  O  W  1- 
X  C  CJ 

>  E  C  CJ 

o  o  

O  £  —  **  *- 
4->  >  4->  TO  O 

—  a. 

^  o  c  >, 

wy-  cu  (- 

C      (-   *J 

qj  to  Q)  £)  CJ 

—  +->  T3  TO  E 
-PC  O  CJ 
TO  CJ  TO  

a—  cd  —  cj 

re  1/1  a  \ 
cd  a.  —  o 

C  -  CJ 

V-  — 

<-  O  CD  O 

CJ  £  o 


265 


Table  C.24 


PERFORMANCE  ON 


INITIAL  PROCESS  FOR  USE  OF  MEDICATIONS  AND  SPECIAL  TESTS  OR 
Pre-  and  Post-PPS  Combined 


PROCEDURES 


Ass  i  gned 


CHF 
%  or  Pat 

Applicable  Applicable 
Points       Patients(2)  ' 


ents  %  or  Patients 

Pat  ients  Pa  t  ients 

Meeting  Applicable  Meeting 

:riteria(3)  Patients(2)  Criteria(3) 


App 1 i cab le 
Patients(2)  Cr 


CVA 

cms  %  or  Pat  i  ents 

Patients  Patients 
Meeting        Applicable  Meeting 

'   |3)  Patients(2)  Criteria(3( 


p02  <  60  Day  i 


p02  <  60  Oay  2 


No  ABCs  and 
Moderate ly  or 
Very  Sick 
(i.e..  ASSC  >=  5 > 
Oay  1  and  2 


i  ntuba  t  ion 


Cardiogenic  shock 
wi  th  End  Organ 
Dam ngc , 

Arrhythmics.  AND 
very  sick  (i.e., 
ASSC  >=  7) 
Oay  1  or  2 

Ca  rd  i  ogen  i  c  shock 
wi  th  Cnd  Organ 
Damage , 

Ar rhythm i cs ,  AND 
Very  s  ick  (i.e., 
ASSC  >=  7) 
Day  1 

Cardiogenic  shock 
w  i  th  End  Organ 
Damage , 

Arrhythmics.  AND 
Very  sick  (i.e., 
ASSC  >=  7) 
Oay  2 

Vent  r i  cu I  a  r 
Oay  1 


Pressors  or  IV 
un I oad  i  ng  agents 
Day  1 


Pressors  or  IV 
unloading  agents 
Day  2 


Ant  i  a  r rhythm  i  c  s 


Th  i  rd  deg  ree 
heart  block 
Oay  1  or  2 
AND  Anterior  AMI 

Th  i  rd  deg  ree 
heart  block 
Oay  1  or  2 
but  not 
Anterior  AM  I 


Congestive  hear 
Ea  i lure  but  no 
shock  Day  1 

Congestive  hear 
Ta  i I ure  but  no 
shock  Day  2 


Chest  pa  in  noted 
by  MD  or  RN  Oay  2 
and  no  IV 
unloading  agents 
Day  2 

Chest  pa  i  n  noted 
by  MD  or  RN  Oay  1 
and  2  AND  no 
evidence  or  shock 
or  congestive 
hea rt  ra  i I ure 
Day  1  and  2  ANO 
no   IV  un I oad  i  ng 
agents  Day  2 

Chest  pa  i  n  noted 
by  MD  or  RN  Day  1 
and  2  ANO 
evidence  oE  shock 
or  congest  ive 
hea  r  t  Ea  i I u  re 
Day  1  or  2  AND 
no   IV  un I oad  i  ng 
agents  Oay  2 


T  >  38.3  C 


Oil 


ing 


St  roptok  i  nasc 
g  i  ven  wi  th  ■  n  6 
hours  oE  time 
symp  toms    t  s  t 


IV  un I oad  i  ng 
agents , 
I ong-ac  t i  09 
nitrates,  oral 
or  IV  B  b I ocke  rs, 
OR  Ca  channc I 
blockers  Day  2 


Long-act  i  ng 
n i  tra tes,  ora I 
or  IV  B  blockers, 
or  Ca  channel 
blockers  Oay  2 


Long-ac  t  i  ng 
nitrates  or 
Ca  channel 
blockers  Day  2 


Sputum  gram 
sta  i  n  qua  I i  ty 
noted 


<   120  m in 
120-239  min 
240-359  min 
360-479  min 
480-719  min 
720-1439  mm 
1140-2880  min 
>  2880  min  or 
No  Ant  i  b i  ot ic 


I mmunocomp  rom  i sed 


<  120  min 
120-239  min 
210-359  mm 
360-179  min 
180-719  min 
720-1439  min 
1440-2880  mm 
>  2880  min  or 
No  Ant  i  b  iot  i  c 


(1)  DeEines  patients  to  whom  process  criteria  are  applicable. 

(2)  %  of  patients  to  whom  criteria  are  applicable  is  the  Traction  or  patients  with  each  disease  who  Eit 
the   listed  deEinition  oE  patient  subsets. 

())  %  oE  applicable  patients  who  meet  the  process  criteria. 
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Table  C.25 

PERFORMANCE  ON  DAY  3    PROCESS  CRITERIA  (1) 
(Pre-  and  post-PPS  combined) 


Process  Scale 
and  Patient 
Subset  (2) 


Chest  X-ray  & 

F. I ec t  roca  rd  i  ogram 

Moderate ly  or 

very  sick 

(i.e.,  ASSC  >=  5 


Process  Cri 


508  assessed 
Lung  assessed 
Chest  pa  i  n  assessed 
MO  note 

Note  on  der  i  c  i  t 


SOB  assessed 

Chest  pain  assessed 

PVCs  assessed 

Nut r i  t  ion  assessed 

Pup i I s  assessed 


App I  icab le 


Pat  ients 
Meet  i  ng 
Criteria 


Pat  ■  ents 
Meet  i  ng 
Criteria   ( t* ) 


Patients 
te  Meeting 
(2)     Criteria  (3) 


Pat  ients 
Meet  i  ng 
Cr i  ter i  a( 


Very,  moderately,  E 
or  somewhat  sick 
( i.e. .  ASSC  >=  3) 
or  chest  pa  i n  noted 
by  MD  or  RN 

Laboratory 

Assessment 

Somewhat  sick  A 
(i.e..   ASSC  A 


Moderately  or 

very  sick 

(  i.e.,  ASSC  >■ 


5) 


Moderately  or 
very  sick 
(i.e..  ASSC  >= 
Chest  pa i n  noted 
by  MO  or  RN 

Modera  te  I  y  or 
very  sick 
(i.e.,  ASSC  >= 


5) 


Cardiac  enzymes 
Ca  rd  i  ac   i  soenzymes 
Arteri  a  I  b lood  gas 

At  least  U  or  7  labs 
At   least  3  of  7  labs 


Use  or  ICU  or 

te lemet  ry 

Somewhat  sick 
(i.e.,   ASSC  3-U) 


7)  Telemetry 


Not  very  sick  M 
(i.e.,   ASSC  0-2)  T. 
on  day  3  and  shock, 
congestive  heart 
fa  i  lure,  or  ASSC 
>=3  on  day  2 

Not  very  sick  M 
(  i.e.,  ASSC  0-2)  Ti 
on  day  3  and 
evidence  of  AMI 

Not  very  sick  H 
(  i.e.,  ASSC  0-2)  T. 
on  day  3  and  no 
evidence  or  ami 

Very  sick  I 
(i.e.,  ASSC  >=tl)  T 
on  day  3 ) 


Use  or 
Med  ica t  ions 
IT  p02  <  6' 


Use  02  or  inttibat 
Use  digital  is  or 


Moderately  or 
ve  ry  sick  (i.e. 
ASSC  >=  5)  and 
S8P  <  90  Day  3 

Vent  r  i  cu I  a  r 
tachyca  rd  i  a 
day  1  or  2  or 
PVCs  Oay  3 


Use  pressors  or  IV 
unloading  agents 
day  3 


Use  antiarrhythmics 
or  oral  or  IV  6 
blockers  day  3 


Use  IV  un I oad  i  ng  5 
agents,  long-acting 
nitrates,  oral  or  IV 
6  blockers,  or  Calcium 
channel  blockers 
day  3 


Chest  pa  i  n  noted 
by  MD  or  RN  day  3 
and  no  IV 
unloading  agents 
Day  3 

Chest  pain  noted 
by  MD  or  RN 
day  2  and  day  3 
and  no  evidence 
of  shock  or 
congestive  heart 
ra  i I ure  day  1  or  2 
and  no   IV  un I oad  i  ng 
agents  day  3 


ing 


se  long-acting 
n  i  tra  tes,   ora  1  < 
IV  B  blockers,  i 
Ca I c  i urn  channe I 
blockers  Day  3 


Chest  pa  i  n  noted 
by  MO  or  RN 
day  2  and  3  and 
evidence  of 
shock  or 

congestive  heart 
ra  i  I  ure  day  1  or 
2  and  no  IV 
un I oad  ing  agents 
day  3 


Use   I ong-ac  t  i  ng 
r>  i  t  ra  tes,  or  Ca  I  c 
channe I  b  I  ocke  rs 
day  3 


(1)  There  are  no  day  3  process  criteria  ror  nip  fracture  patients. 

(2)  Defines  patients  to  whom  process  criteria  are  applicable. 

(3)  %  of  patients  to  whom  criteria  are  applicable  is  the  Traction  or  patients  with  each  disease  who  fit  the  listed  definition 
of  patient  subsets. 

(tt)  %  of  applicable  patients  who  meet  the  process  criteria. 


267 


CO 

EN 

— 

i— 

< 

CL 

Li. 

o 

CO 

H 

LxJ 

co 

CD 

co 

 i 

<  UJ 

—  cc 

o  35 

0) 

Lj   | 

c 

0-  — 

CO  < 

-Q 

U- 

e 

cc 

o 

O  h- 

o 

SO 

u.  cc 

CM 

< 

CO 

<  Ld 

CL. 

O 

—  X 

CL. 

cc 

1 

CU 

Ld  LU 

4-5 

— 

h-  > 

t/5 

o 

TO 

CC  I— 

a 

h- 

O  CO 

Ld 

T3 

CO  O 

CO  Z 

TO 

Ld  O 

o  o 

o 

a> 

CC  X 

L. 

CL  1— 

a. 

-i  St 

< 

z 

o 

r- 

o 

Q 

< 

Z 

O 

CE 

z 

< 

CC 

o 

u. 

CC 

Ld 

a. 

ro 

c/) 

c  c  — 

C0 

0)  —  s- 

 JJ  QJ 

4->   Q)  4-> 

TO  QJ  — 

0-  S  «- 

cj 

— 

cvj 

Q)  — 

.O  1/3 

TO  4-5 

in 

o  c 

—  cj 

Q.  4J 

Cl  TO 

<  Cl. 

1 

O  tn 

•-  4-5 

—  qj  c 

a  

Q..O  o 

<  ro  a- 

T3 

Q)  1/5 

C  4J 

cn  c 


4-3 

QJ 
U3 

n 

to 


(\J  (M  vO  CM  \0  C\J 


in  ■—  \0  O  fO  CM 

r-  co  \o  \o  in  \o 


r-  O  O  J-  oo  \0 


ON  in  vO  in  ro 
-3"  f--  ro  .J  .d"  ro 


ro  t—  00      CO  ro 

in     cm  .3-  cm  cm 


ro  i—  in  VO  CM  ro 
in  r~-  ro  J-  CM  CM 


in  O 

i/l  Cl. 

>> 

< 

TO 

■a 

ro 

C 

cu 

4-: 

JZ 

TO 

4-> 

> 

03 

in  Q) 

TO 

to  s_ 

o 

o 

CU  3 

o 

CJ 

CJ  </! 

O  TO 

4-5 

o 

S-  CU 

c 

CU 

q-  s: 

< 

CC 

00 


II 

A 


35 

co 


o  o  >-  in 
in  m    ;f  i- 

r-     .  O 

m  ■  ii  n 

>>  II       CM  A  A 

<-  12  A  II 

O  TO        A    II    L.  S_ 

4->  —  1-       A  O  O 

TO  O  <- 

—     —     o    in  o 
^     too    o  <-  •  o 

-O  E  ro  O  — -      ro  ro 

E  S-  r-    .  V 

TO  O       ro  —       V  V 

C  C  V       TO  ro 

o    .a     v  c  o  cj  i — 

C        TO  TO        0  CJ  CO  CJ 
ZiCCIJI 

e*.  L- 


(/3 
>> 
TO 

•o 


4J 


4-3 
O 
0) 
1_ 
c 
o 

CJ 


0} 

2 


TO 

> 


o 
o 


>> 

s_ 
o  o 
4-1  in  in 


33 

co 


!_ 

O  r- 


O 

m  •  ii 

\0  A 

II 

All  S- 
A  O 

i_ 

o  s-  in  in 

CJ  r-  1-  o 
E  ro  O  —  ro 

i.  T-       ■  V|| 

O       ro  —       A  V 
C  V       TO  ro 
n       V  C  O  CJ  h- 
TO  TO       CD  CJ  CO  CJ 
ZXCCI  Jl 


TO  O 


ru 

(/3 

TO 

CU 

in 

CM  f —  r-  r- 

— 

■D 

J"  vO  ro  \0 

CM  CM  ro  r- 

J3 

O 

CO 

QJ 

r* 

4-> 

— 
> 

j-  d-  j-  J- 

(A 

•  .  . 

4-> 

CO  CO  CO  CO 

c 

CU 
— 

4-> 

TO 

Q. 

(t- 

o 

c 

ro  ro  ro  ro 

o 

— 

4-) 

•  CJ 

CJ  TO 

—  !_ 

ro  CM  r-  O 

X!  <>- 

ro 

TO 

CU 

o  cu 

!_ 

-  c 

—  JZ 

0J 

CC  ■- 

—  4J 

4-5 

X  s- 

a 

O  3 

a  v> 

i_ 

TO  — 

CJ 

•  •  -o 

a  c 

CU  OJ 

(fl 

3  TO 

!  

V) 

1 

TO  JO 

QJ 

J£  » 

TO 

U5 

O 

s_  CP 

TO  O 

4-> 

o 

o  s- 

CU 

i. 

!  

U5 

Cl. 

4-> 

QJ  Q.J3 

c  — 

4->  a  3 

OJ 

O  3 

ro  ro  ro  ro 

■-  TO 

W  J3 

—  o 

OJ 

t_ 

4-5 

4-> 

s_ 

c*.  u-  U.  d_ 

O  OJ 

4-5 

O  T3 

3 

O  O  O  O 

1- 

c 

4-5 

OJ  o 

4J 

i/>  ro 

03 

QJ 

<~  o 

— 

ro  CM  r-  O 

in 

OJ 

c  — 

3 

ru  ro 

4-5 

E 

—  JO 

O 

o  — 

TO 

O  s- 

a  o 

s_  CJ 

JZ 

d  4->  U- 

o 

E  s- 

(/) 

O  O 

c 

4-5 

JZ 

o 

c 

>  B 

QJ 

o 

4-5 

O  J3 

4-5 

4^  > 

c 

CO 

CL. 

y)  O  <*- 

4->  4-> 

OJ 

OJ 

c 

T3 

QJ  1/5 

n 

CJ 

—  4->  T3 

TO 

4->  C 

OJ 

O 

ro  L. 

ro  QJ 

•  CU 

a.— 

t/3 

00  4-> 

4-> 

a 

ro  to 

1/5  TO 

a 

oj  a 

< 

A 

c 

QJ 

!_ 

—  u-  r.  u- 

S-  O 

<4-  o 

4J 

o 

CU 

QJ 

>  I- 

Q  4-ifeS 

cu 

<*-  >> 

<t- 

ro 

t-  CM 

CO 

>  a 

o 

OJ 

z 

268 


ni 

ro 

m 

V; 
CD 

Oi  CO 

CO 

c  ■  ~ 

J  SO  C7N 

u>  lli  O 

QJ 

 i_ 

lA  CO  CM 

a)  *-> 

r-  \o 

ip\  Is* 

TO 

CD  •— 

o 

n 

t 

CO 

o 

CD 

\— 

Z. 

CD  C\J 

> 

n  to 

^ 

<o  -u 

CO  -J 

o  c 

*    *  " 

*  * 

—  a) 

L"  w  C\J 

c 

CD 

O 

a  j-> 

a.  co 

CO 

<  a- 

CO 

1 — 

Q. 

UJ 

CO 

1 

c*— 

CO 

O 

io 

O 

to 

0)  c 

c 

Z. 

a 

r-  r—  r— 

o 

— 1  o 

<£.  — 

< 

CO  0- 

—  1— 

•  o 

o  o 

d) 

■o 

CD  co 

UJ  CC 

c 

a> 

—  s_ 

■~ 

C  CO 

r-  r-  r— 

r—  f—  t— 

rt  c^ 

CO 

co  u— 

co 

Z 

£ 

•-  c 

O  CD 

t_ 

cc  — 

O 

to  — 

— 

CD 

o 

o 

10  O 

t/5 

—  4J 

4J 

Li_   1 

<  a- 

>> 

CO 

Q. 

< 

CO 

TO 

>> 

Q.  CO 

L. 

<t  — 

"O 

CO 

co  — 

u 

—  o 

"O 

cc  cc 

1 

rO 

CD  CD 

(/) 

UJ  <L 

4-> 

ro 

!  

(0 

h—  O 

to 

C 

CO  J3 

0} 

ill 

—  o 

o 

*~ 

C 

CO 

to 

o 

cc  >- 

Q. 

JZ 

*~ 

co  O 

JJ 

o 

f) 

u  s 

+-J 

JZ 

CD 

s_ 

+^ 

!  

to 

a 

1 

CO  UJ 

c 

-* 

*~ 

CD  o.n 

CO  f— 

TO 

to 

? 

•p  a 

CD 

UJ  ID 

CO  CD 

Jl 

—  co 

to  JZ 

O  O 

1 

CD  S- 

o 

4-3 

!_ 

+j 

O  < 

O  3 

CD 

o 

O  CD 

O  co 

JZ 

CD 

c 

4J 

Cj_  X 

Q. 

s-  CO 

o 

^" 

to  CO 

CD 

CD 

1— 

Q.  Q) 

to 

CD 

 1  — 

s: 

CC 

o 

Q>  CO 

4-> 

E 

<  Z£ 

o 

O  — 

CO 

z 

O  5- 

a  o 

O 

1_  CD 

JZ 

a  *-> 

c*. 

\— 

o 

CO 

o 

E  5- 

to 

o 

I — 

o  u 

c 

4J 

o 

II 

CD 

JZ 

o 

C 

<! 

A 

Z}  A 

5  E 

CD 

o 

z 

z 

co  z 

O  JZ 

4-> 

o 

Z5 

>  Z3 

4->  ? 

c 

CO 

CO 

CD 

a 

UJ 

>> 

co  O 

o 

^* 

4-1  4-> 

0) 

a) 

z 

o  o 

o  o  o 

C 

"O 

4J  LPs  if^ 

CD  tO 

13 

r-    .  o 

co  «—    •  O 

—  4J 

"D 

ro 

cc 

lA  • 

S_       Lf\  . 

4->  C 

CD 

O 

o 

II  CM 

on  \o 

CO  CD 

*J 

u_ 

J3  A  II 

£1  A  II 

a— 

(0 

cc 

CO      A  II 

CO       A  II 

a 

UJ 

—  i-  A 

—  1-  A 

to  CO 

a 

o  s- 

O  S- 

cd  a 

CO 

-        O  «- 

-        O  5- 

c 

CD 

roO  O 

co  O  O 

—  c^  jz  <*- 

E  ro  o  ~ 

<»-  o 

4J 

o 

s_  .—  . 

1-  t-  . 

CD 

O      ro  — 

O       ro  — 

4-> 

C  V  co 

C  V  CO 

<D 

JZ       V  C 

JZ      V  c 

Ci_ 

10 

co  co  (D 

co  co  CD 

ro 

XJ 

z  ^:  cr 

o 

z 

c»- 

c<- 

in 

to 

269 


PO 

4-)  cn  a: 

c  c 

A.  1 

/-~\  (v-i  fti 
U  WJ  l\J 

LA  C\J  C\J 

rO  \0  r*** 

O  CT\  \0 

4-J   Q)  4-J 

r*-  r —  ^0 

-3" 

TO  CD  •  — 

\/J 

l_ 

LU 

CD  C\J 

JO  C/3 

r- 

ro  Cn  -J 

n 

U  L_ 

■— 

Cjn  J 

C7N  -3"  C\J 

1 1 

*~ 

»—  i—  i— 

UJ 

<<  u_ 

1 — 
UJ 

f  A 
<JJ 

1 

UJ 

ID 

4-J 

tf\ 
\J) 

—  DC 

^-   f—  T— 

^—  ^*  t— 

j 

d  j_j  o 

-< 

TD 

<3!  to 

CD 

C 

1 1  | 

Q_ 

CD  to 

' 

f—  i—  t— 

r-  i—  »— 

C  4-3 

<< 

O 

CTl  C 

LL.  — 

o 

O  3 

c/j  O 

(/) 

i—  u 

00 

*o 

SI 

< 

TO 

>1 

*<.  — ' 

"O 

TO 

—  LlJ 

1 

"D 

—1  2T 

4— ' 

CO 

ro 

h- 

o 

c 

—  X 

a 

a; 

•— 

c 

*^ 

'"" 

o  ~~ 

"D 

TO 

4-J 

-C 

4-J 

oo 

TO 

— ' 

■>> 
-> 

00 

> 

LJ 

1 

(/)  CD 

TO 

O 

0 

^  _^ 

o 

o 

t-? 

QJ  13 

o 

0) 

o 

QC 

Q. 

o 

JZ 

O  TO 

o 

^~ 

i-  QJ 

c 

QJ 

—J 

Q-  S 

<! 

o 

o 

r— 

o 

~ 

CO 

o 

Q 

Q 

II 

03 

A 

3  A 

^ 

CD  ^ 

o 

>  3 

CO 

CQ 

UJ 

>> 

a 

L. 

o  o 

o  o  o 

< 

ir\  in 

4->  ^  in 

21 

r-    .  o 

CO  f-    .  o 

cr 

Lf\  • 

S.      la  • 

o 

>> 

II  CM 

O  II  VO 

U_ 

-QAM 

13  A  II 

cc 

o 

TO       A  II 

CD       A  II 

UJ 

4-J 

—  S-  A 

—  S-  A 

TO 

o  s- 

O  S- 

-        O  5- 

-       O  <- 

TOO  O 

CD  O  O 

JO 

e  ro  o  «— 

E 

i—  r-  . 

4.  r-  . 

TO 

O      po  — 

O      co  — 

4-J 

C 

C  V  TO 

C  V  CD 

03 

o 

-Q       V  C 

n     v  c 

t/> 

c 

TO  TO  CD 

CD  CD  Q) 

n 

Z  ^  CC 

z  ^.  cc 

<*- 

00 

CO  S3-  00 


caj  t-  m 

J-  CM  co 


4J 
C 
CO 

4-1 

CD 

a 

*D 

CD 

N 

—  to 

CD  CD 
4->  E 

a  4-> 

O  CM  CD  CD 

£      o  > 
II  C  CD 

—  A  O  Z 


CM  «- 

CO 

CO 

CO 

CM  O 

O 

00 

co  co 

MD 

J- 

J- 

O  O  O      J"  3" 

COO 

r-  t-  t-      00  CO 


CM 


CM  CM  CM       r-  r- 


CNJ  r-  O      i-  i- 


ITl 


lA 


o 


o 


CD 

s_ 

<4-  T5 

Et- 

!_ 

O 

a> 

(/> 

1/5 

4-1 

in 

c 

o 

O  "D 

E 

—  CD 

V) 

CO 

4-1 

O 

4J  V) 

O 

>> 

4-1 

o 

s_ 

o  — 

4-J 

CD 

1/1  TJ 

Q. 

—  E 

c 

■a 

CO 

!_ 

CD 

E 

O 

CD 

o 

c 

E  4-1 

0 

—  S. 

o 

4-1 

CD  O 

4-1 

CJ 

4J  Q. 

4-> 

S-  C 

c 

c 

C  E 

CO 

.Q 

CO 

c 

I/] 

CO  O 

(A 

a 

O 

>> 

4-1 

o 

4-1 

C 

c 

E  4-> 

CD 

E 

c 

CD  O 

c 

.c 

CD 

Ol  E 

0) 

Ol  c 

0 

Ol 

CD 

CO 

C 

4->  — 

C 

C  3 

4-1 

4-1 

3  CD 

CO 

C 

4-1 

CD  E 

4-1 

01 

£3 

s_ 

a  3 

.C 

c 

T  -Z  E 

(0 

O  O 

CO  IT 

o 

z 

ao  — 

a 

o 

CO 


00 
co 


CD 
> 
CD 
d- 

4-1 

c  t- 

0)  CD 

4-1  4-1 

I/)  CD 

co 

1_  s_ 
CD  O 

a 

■3" 

1_ 

o  >> 

CD 

>  T3 

CD 

z 


01 


3 

o 

s_ 

o 

4-1 

c 

CD 

in 


a 


o 

co 


00 
CO 


01 

> 

01 


4-1 

c  t- 

CD  CD 

4-1  4-1 

ui  co 

00 

(_  s_ 

CD  O 

a 
i_ 

o  >> 

CO 

CD 


4-1 

C*_ 

Q 

> 

CD 

00 

CD 

CD 

S 

4-J 

> 

t0 

0) 

Jj 

TO 

i-i 

rk 

c 

o 

~ 

•  o 

CD  CD 

  ^_ 

J2  c*_ 

CO 

CD 

t  \  n\ 

•--  Z 

tu 

1  1 

Q.  00 

CD 

o 

Qj  Qj 

i  

C/3 

co  -Q 

01 

CD 

o 

CO  O 

c 

s  

ex. 

cd  a. 

■u  a 

—  CO 

-IT 

s_ 

O  CD 

00 

s_ 

oo  co 

a> 

Ul 

00 

oo 

01 

CD  CO  -O 

O  — 

O  5- 

</> 

U 

i-  CD 

-CI 

a  4J 

4-1 

■> 
> 

C 

E  «- 

01 

o  o 

4J 

4-1 

_ 

>  E 

CO 

O 

a. 

~ 

O  -C 

4-J   >  <4- 

CO 

0 

a. 

oo  o 

4-1  4-1 

c 

a> 

c 

o 

0  00 

JQ 

—  4-1 

4-1 

CO 

■P  C 

o 

co  CD 

C 

Q.— 

*J  o^ 

Ql 

00  CO 

01 

Q. 

CD  Q.T3 

CO 

C 

—  0_  "D  t- 

u-  o 

0) 

o 

CD 

4J 

O  &5  oo 

i-  CM 

CO 

01 

270 


to 

H 

EN 

< 

Li_ 

O 

to 

UJ 

co 

co 

CO 

 1  h- 

<  z 

—  UJ 

CD 

CJ  Q 
UJ  — 

c 

a.  o 

JO 

to  o 

E 

< 

O 

01 

o 

o  cc 

u.  < 

CO 

ON 

—I 

0_ 

C\J 

<  =1 

n 

-  O 

1 

cc  to 

+^ 

LjJ  < 

t/5 

CD 

I-  > 

O 

~~ 

—  O 

Q. 

-Q 

cr  cc 

03 

cj  co 

\— 

UJ 

C 

to  CC 

TO 

to  UJ 

UJ  O 

1 

o 

0 

O  X 

<_ 

Ol  I— 

Q. 

D_  — 

_l 

< 

z 

o 

r- 

Q 

Q 

< 

z 

o 

CE 

z 

< 

s: 

cc 

o 

u. 

cc 

UJ 

00  CD 

00  S- 

CD  3 

O  V> 

O  CD 

S_  (1) 

a.  2 


4-> 

CD 

00 

JO 

to 


. — . 

ro 

10 

— 

Ol  TO 

c 

C  — 

J-  00  f-  LA  LA  ro 

CD 

—  i. 

—  rf\ 

4-) 

CD  4-> 

r—  r—  la  la  ia  la 

TO 

CD  — 

0. 

CJ 

CD  CO 

XI  00 

TO  4-> 

vO  fO  C\J  ro  0\  O 

o  c 

—  CD 

LA  CM  f-  CM  00  00 

• —  t~  r~-  ' — 

a-u 

Q.  TO 

<  Q- 

1 

o 

00 

0)  c 

CI 

Q.J0  O 

< 

TO  CL 

T3 

CD  00 

C  4-> 

ai  c 

oo  o 

00 

oo  Q- 

>> 

< 

TO 

JZ 


O 
CD 

CJ 
CD 
01 


00 


II 
A 


CO 
CO 


o 

LAlA 


-QAM 
TO  A 

—  L. 

O  S- 

—  o 

TO  O 
EnO 
s-  i—  . 
O  ro 
C  V 
JO  V 
TO  TO 
z  ^ 

t)- 


O  i-  lA 

-3"  i- 

O 

•  II  II 

CM  A  A 

II  !-  S- 
A  O  O 

t-lAO 

o  «-  •  o 

. — .     ro  ro 
V 

—  V  V 
TO  CO 

C  O  CJ  r- 
OOCOO 
CC  X  "3-  X 


Os  ro  CM  CM  r-  O 


lA  O  i—  On  lA 
J"  r»  J"  J"  CM  ro 


SO  ro  00  ro  J"  On 
LA  CM  r-  CM  00  r- 


oo 
>> 

TO 
TO 

ro 

C 


o 

<D 

t- 
o 
o 


o 
o 

CD 

3  A 

TO  Z 
>  =5 
CO 

>> 

O  O  Or 
P  LA  lA  J" 
(0  r-    •  O 
S-       LA    •  II 
VO  A 

II 

A   II  S- 


O  II 
JO  A 
TO 

—  t_ 

O  1- 

—  O  ^-  LA  LA 
TOO       O  r-  I-  O 
Ecoo—  ro 
(-  r-    .  VII 

O      ro  —       A  V 
C  V        TO  ro 
JO       V  C  O  O  h- 
TO  TO       CD  CJ  CO  O 


CO 
W 
(0 
CD 


JZ 

o 

TO 
CD 

jz 

> 

oo 
P 
C 
CD 

■P 

ro 
a 

o 

c 
o 

p 
•  o 

CD  TO 

—  1_ 

JO  <t- 

ro 

o  CD 
■-  jz 

—  p 
a. 

Q.  00 
TO  — 

CD  CD 


!_ 
CD 
P 

!_ 

o 

oo 

00 

a> 
o 
o 

L. 


TO  JO 

TO  OO 

TO  O  P 

  CD 

!   00  0. 

a>  ajo 
■u  a  3  <D 

—  TO  00  JZ 
t-  4J 

O  CD  P 

S-  C  -P 

oo  to  CD  CD 

oo  —  CD 

CD  TO  P  E 

O  —  TO 

o  s-  a  o 

s_  CD  JZ 

o 


E 

o 
jr 

5 


o . 


ro 


c 

—  a. 

oo  O  <<- 

*J  j->  CD  CD 

C  XJ  — 

CD  00  JO 

—  4J  X)  TO 
•P  C  CD  O 

ro  CD  -P  — 

Q. —  00  — 

+j  —  a 

oo  to  —  Q. 

aj  o.  < 

C  CD 

—  t>-  jz  t^ 

0-  o  4->  O 

CD 

Q  S-5  -P&S 
r-  CM  ro 

jr 


271 


o 

ro 


0 


JO 

CD 


CD 

CO 

ro 

CD 

co 



CD 

4-> 

CJ1  CD 

co 

c 

C  — 

ON  ON  O  r-  VO 

ro  C\J  J-  00  VO 

CM  \D  J3-  00 

CD 

—  1_ 

TJ 

(- 

4->  CD 

i-      vo  CO  r- 

3"  CO  h- 

z 

4-1 

CD  4-1 

r*-  r—  *o  \o  co 

.3"      ro  JT  LA 

r—  CM  ro  t— 

JZ 

uj 

CD 

CD  — 

o 

Q- 

S  5- 

CO 

h- 

o 

CD 

< 

a. 

th 

U_ 

0  CM 

o 

JO  CO 

c/) 

CD  •*-> 

vO  r-  J"  .-  V£> 

vO  «"■  CM  \0 

r—  i —  i —  i — 

CO 

ET 

O  C 
—  Q) 

j-  co  cP\  1— 

-3-  CO  t—  ro  i— 

o  o  o  o 

nt 

i—          <—  lA 

CD 

LLJ 

Q.4-) 

ZD 

Q.  CD 

4-1 

UJ 

<  Q. 

CD 

a 

1 

— I 

1 

c<- 

o 

co 

O 

o 

4-1 

UJ 

T3 

CD  C 

c 

a; 

a 

ro  ro  ro  ro 

O 

C/} 

c 

Q.JO  o 

CD  Ci- 

4J 

JO 

•  o 

o  cd 

E 

CD  CD 

U.  ID 

o 

"O 

—  !_ 

r— 

o 

CD  co 

^  ,  

ro  CM  ■—  O 

JO  <- 

00 

U  4-1 

CD 

—  < 

</) 

en  c 

0) 

O  CD 

£_ 

Cd  ui 

a. 

»  c 

—  JZ 

01 

UJ  U- 

a. 

"n 

CO 

cc  — 

—  4J 

4-1 

1 — 

I 

co  c? 

>> 

EA 

X 

a 

j-s 

<3. 

CD 

>> 

O  Zi 

a.  co 

i_ 

UL  — 

co 

"D 

CD 

CD  — 

o 

O  X 

DO 

rO 

T3 

•  •  T3 
Q.  C 

CD  0 

(0 

C/3  X 

ro 

3  CD 

ro  ro  ro  ro 

!  

UJ 

CO  t— 

"a 

c 

1 

CD 

CD  JO 

0 

LJ  — 

c 

C 

_i  . 

!_ 

U-  U-  U- 

CD 

CO 

o 

CD 

JZ 

S-  CD 

O  O  O  O 

CD  O 

4-1 

o 

O 

4-1 

JZ 

O  1- 

0 

s_ 

C£ 

1 

4-> 

ro  CM  i—  O 

s  

co 

a. 

^~ 

CD 

4-1 

CD  O.JO 

l_ 

cz  — 

o 

4-1  Q.  3 

0 

-J 

Q. 

CO  CD 

O  3 

—  CD 

CO  JZ 

< 

co  t- 

o 

4-1 

—  o 

L. 

4-) 

CD  3 

CD 

u 

4-1 

O  0 

o 

O  co 

JZ 

0 

O  "O 

!_ 

c 

O  CD 

o 

l_ 

CD  O 

CO  CD 

0 

0 

h- 

s_  CD 

CD 

s_ 

CO 

0 

q-  s: 

o 

c  — 

0  CD 

4-1 

E 

Q 

o 

—  co 

O  — 

CD 

Q 

O 

Q.  O 

< 

1_  CD 

JZ 

,  , 

Q.  4-1  C>_ 

Z 

T— 

o 

C 

o 

UJ 

CO 

II 

o 

T— 

CD 

CM 

E  i- 
O  O 
JZ 

on 

nts 

o 

A 

3  A 

>  E 

0 

z 

O 

4-1 

< 

Z 

CD  Z 

O  JZ 

4-1 

s: 

Z3 

>  Z5 

5  c 

ro 

cr: 

CO 

co 

CO 

a. 

o 

>> 

>, 

co  o  '<- 

Li. 

s_ 

s.  c_ 

CD 

4->  4-1 

0) 

0 

tr 

O       O  LA 

o  o  o 

E 

T5 

UJ 

la  la  i- 

4J  lA  LA 

CD  CO 

JO 

O- 

r-    •  O 

CD  <~     •  O 

o  a 

—  +J  T3 

CO 

lA    •  II 

t-       la  • 

o 

4-1  CZ 

0 

o 

II       C\J  A 

on  vo 

ro  4-1 

CD  0 

JO  A  II 

JO  A  II 

■  CO 

Q.— 

<s> 

CD       A   II  S- 

CD       A  II 

CO  O 

4J 

a 

—  i-  AO 

—  t-  A 

ro  Q. 

CO  CD 

a 

O  I- 

O  S- 

0  a 

(0 

—       O  S_  O 

—       O  S-  LA 

a  ai 

cz 

aj 

CD  O      O    •  O 

CD  C       O  r-  O 

E 

—  <i-  JZ  d- 

Eno-roro 

E  ro  O  —  ro 

s_  — 

Ct-  O 

4-> 

o 

l_  i—  . 

<-  r-    .  || 

CD  T3 

0 

O       ro  —  V  V 

O      ro  —  A  V 

>  3 

Q&S4J&5 

4-> 

C  V  CD 

C  V  CD 

CD  — 

CD 

jo     v  c  o  i— 

J2       V  CUh 

<~  CJ 

C|- 

CO 

CD  CD       0  CO  O 

CD  CD        CD  CO  O 

X 

i-  CM 

ro 

jo 

z     cr  ^  x 

>UJ 

o 

3 

c>- 

CD 

JZ 

z 

5 

272 


Tab  I e  C. 31 

PERIOP  PATIENT   PROCESS   (SURGERY   PATIENTS  ONLY) 
FOR  HIP   FRACTURE  PATIENTS 
(Pre-  and   Post-PPS  Combined) 


H  I  P 

%  or  Pat  ients 
App lie-  Pat  i  ents 

Process  Assigned     able  Applicable  Meeting 

Subsets! l )  Measure  Points      Points      Patients(2)  Criteria(3) 


All   Su  rg  i  ca  I   Pa  t  i  en  t s 


Surgical    Patients  with 
Temperature  <  101   F  (38.3  C)  on 
postop  Days  2,3  &  4,    no  postop 
hip  infection,  no  preop  or 
postop  pneumonia,   and  WBC  <  15 
or  not  tested  on  postop  days 
2,  3  &  U 

Surgical   Patients  treated  with 
internal    fixation  or  pinning 

Surgical    Patients  treated  with 
a  rth  rop I  a  sty 


All    Surgical  Patients 


Surgical   Patients  with  Chest 
Pain  or  shortness  of  breath 
preop  on  day  prior  to  surgery 
or  day  of  surgery 

Su  rg  i  ca  I   Pa  t  i  en  t  s 

Very  sick  (i.e.,   ASSC  >=3) 

on  postop  Day  2 

Surg  i  ca  I   Pa  t  i  en  t  s 
not  very,    somewhat,  or 
moderately  sick 
(i.e.,   ASSC  =  0-2) 
on  postop  Day  2 

Su  rg  i  ca I    Pa  t  i  en t  s 

Very  sick  (i.e.,  ASSC  >  =  3) 

on  postop  Day  1   or  2 

Su  rg  i  ca  I    Pa  t  i  en  t  s 

Very  sick  (i.e.,  ASSC  >=3) 

on  postop  Day  1   or  2 


Surg  i  ca  I   Pa  t  i  ents 
Very  sick  (i.e.,  ASSC 
on  postop  Day  1 


=  3) 


Su  rg  i  ca  I    Pa  t  i  en  t  s 

Moderately  sick  (i.e.,   ASSC  =2) 

on  postop  Day  1 


Su  rg  i  ca  I   Pa  t  i  en  t  s 
Very  sick  (i.e.,  ASSC 
on  postop  Day  2 


=  3) 


Pa  rente  ra  I 
Antibiotics 

No  pa  rente  ra  I 
antibiotics  on 
postop  days  1, 
2,  3  &  4 


Hip  x-ray  day  o.f 
surgery 

Hip  x-ray  day  of 
surgery  or  postop 
day  1   or  2 

Hema  toe  r  i  t  (or 
Hemog I ob  i  n )  Day  of 
Surgery  or  postop 
Day  1 

EKG  Day  prior  to 
surgery  or  day  of 
s  u  rg  e  ry 


BP  measured  at  least  5 
times  on  postop  Day  2 


BP  measured  at  least  3 
times  on  postop  day  2 


CXR  or  EkG  on  postop 
Day  1  or  2 


At  least  4  of  6  labs  4 
(BUN,  Cr,  Na,  K,  WBC,  HCT ) 
on  postop  Day  1  or  2 


100 
57 


77 
9 


Surg  i  ca  I    Pa  t  i  ents 

Moderately  sick  (i.e.,   ASSC  =2) 

on  postop  Day  2 


ICU  on  postop  Day  1 
telemetry  on  postop 
Day  1 

I CU  on  postop  Day  1 
Telemetry  on  postop 
Day  1 

ICU  postop  on  Day  2 
Telemetry  on  postop 
Day  2 

ICU  postop  on  Day  2 
Telemetry  on  postop 
Oay  2 


58 
39 

100 


96 


100 
67 

90 

78 

66 
76 

38 
65 


Ml 

9 


28 
8 


34 

7 


20 
3 


(1)  Defines  patients  to  whom  process  criteria  are  applicable. 

(2)  %  of  Patients  to  whom  criteria  are  applicable  is  the  fraction  of  patients  with  each  disease  who  fit 
the  listed  definition  of  patient  subsets. 

(3)  %  of  Applicable  Patients  who  meet  the  Process  Criteria. 
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Table  C.U9 

PER  I  OP  PATIENT   PROCESS  (SURGERY  PATIENTS  ONLY) 
FOR  HIP  FRACTURE  PATIENTS 

(Pre-  versus  Post-PPS) 


%  of  Pre  Pa t  i  ents   %  of  Post  Patients 

Applic-  Pa  t  i  ents  Patients 

Process  Assigned       able        Applicable        Meeting  Applicable  Meeting 

Subsets(l)  Measure  Points       Points       Patients(2)     Criterla(3)  Patients(2)  Criteria(3) 


Al I   Surg  ica I   Pat lents 


Surgical   Patients  with 
Temperature  <   101    F   (38.3  C)  on 
postop  Days  2,3  &  U ,   no  postop 
hip    infection,   no  preop  or 
postop  pneumonia,   and  W8C  <  15 
or  not   tested  on  postop  days 
2.3  &  '4 

Surgical  Patients  treated  with 
internal   fixation  or  pinning 

Surgical  Patients  treated  with 
a  rthrop I  a  sty 


All   Surgical  Patients 


Surgical    Patients  with  Chest 
Pain  or  shortness  of  breath 
preop  on  day  prior  to  surgery 
or  day  of  surgery- 

Su  rg  i  ca I   Pa  t  i  ents 

Very  sick  (i.e.,  ASSC  >=3) 

on  postop  Day  2 

Su  rg  i  ca I   Pa  t  i  en ts 
not  very,   somewhat,  or 
mode  ra  te I y  sick 
(i.e.,  ASSC  =  0-2) 
on  postop  Day 

Su  rg  i  ca I   Pa  t  i  ents 

Very  sick  (i.e.,  ASSC  >=3) 

on  postop  Day  1  or  2 

Su  rg  i  ca I   Pa  t  i  ents 

Very  sick  (i.e.,   ASSC  >=3 ) 

on  postop  Day  1  or  2 


Su  rg  ica  I   Pa  t  i  en t s 
Very  sick  (i.e.,  ASSC 
on  postop  Day  1 


>=3) 


Pa  rente  ra  I  3 
Ant  i  b  i  ot  i  cs 

No  pa  rentera  I  2 
antibiotics  on 
postop  days  1, 
2,3  &  14 


Hip  x-ray  day  of  2 
surgery 

Hip  x-ray  day  of  2 
surgery  or  postop 
day   1   or  2 

Hema  toe  r  i  t  (  or  1 
Hemog I ob  i  n )  Day  of 
Surgery  or  postop 
Day  1 

EKG  Day  prior  to  2 
surgery  or  day  of 
surgery 


BP  measured  at  least  5 
times  on  postop  Day  2 


BP  measured  at  least  3  1 
times  on  postop  day  2 


CXR  or  EKG  on  postop 
Day  1   or  2 


At   least  1  of  6   labs  14 

(BUN,  Cr,  Na,  K,  WBC.  HCT) 
on  postop  Day  1  or  2 

ICU  on  postop  Day  1  3 

Telemetry  on  postop  2 
day  1 


100 
51 


59 
38 

100 


96 


72 
9 


100 

614 

88 

67 

66 
70 

52 
66 


149 
2 


100 
63 


57 
39 

100 


96 


83 
9 


100 

71 

92 


66 


65 


35 
15 


Su  rg  i  ca I    Pa  1 1 en t s 
Mode  ra  te I y  sick  (  i  . 
on  postop  Day  1 


ICU  on  postop  Day  1 
ASSC  =2)       Telemetry  on  postop 
Day  1 


Su  rg  i  ca I   Pa  t  i  en ts 

Very  sick  (i.e.,  ASSC  >=3) 

on  postop  Day  2 


Surgical  Patients 
Mode  ra  te I y  sick  ( i . 
on  postop  Day  2 


ICU  on  postop  Day  2 
Telemetry  on  postop 
Day  2 


ICU  on  postop  Oay  2 
ASSC  =2)       Telemetry  on  postop 
Day  2 


29 
3 


143 
3 


21 
1 


27 
12 


25 
1  1 


19 

6 


(1)  Defines  patients  to  whom  process  criteria  are  applicable. 

(2)  %  of  Patients  to  whom  criteria  are  applicable  is  the  fraction  of  patients  with  each  disease  who  fit 
the   listed  definition  of  patient  subsets. 

(3)  %  of  Applicable  Patients  who  meet  the  Process  Criteria. 

10/5/89,    17:17,  ER 
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For  the  aggregated  process  measures,  we  based  our  scale  construction  on  a  Likert  Scal- 
ing model.  This  consisted  of  adding  counts  of  related  activities  that  had  been  performed. 
The  model  assumes  that  each  process  criteria  represents  the  process  concept  imperfectly. 
Under  this  statistical  model,  the  items  will  be  correlated  with  each  other  and  with  the 
unmeasured  conceptual  value.  For  example,  whether  or  not  the  physician  determined  what 
medications  the  patient  was  taking  is  correlated  both  with  whether  or  not  the  physician 
determined  any  past  surgeries  and  with  the  concept  of  initial  MD  assessment.  Our  disease- 
specific  process  scales  are  not  the  same  as  the  unmeasured  conceptual  values,  but  are  corre- 
lated with  them. 

Based  on  the  average  correlation  between  the  items,  it  is  possible  to  estimate  the  corre- 
lation between  the  summed  measure  and  the  underlying  conceptual  value.  This  is  called  the 
inter-item  "reliability." 

The  three  scales  described  are  suitable  for  constructing  reliability  estimates.  Scale  reli- 
abilities for  the  initial  MD  cognitive  assessment,  RN  assessment,  and  lab  process  scales  are 
excellent  for  all  the  diseases  (Table  C.50). 

Reliabilities  range  from  0  to  1  (since  they  are  correlation  coefficients),  and  values  above 
0.5  are  generally  considered  good  for  research  purposes.  If  reliabilities  are  lower,  then  it  is 
easier  for  the  scale  to  measure  peculiarities  of  the  individual  items  that  make  up  the  scale. 
Also,  there  is  less  statistical  precision  to  show  effects  that  PPS  or  other  reforms  may  have 
achieved  through  changes  in  the  process  of  care.  If  a  reliability  is  only  0.5,  the  best 
R-squared  we  could  hope  to  find  in  a  predictive  regression  is  0.25. 

A  spot  check  suggests  other  reliabilities  are  also  good.  For  example,  the  two-item  initial 
chest  X-ray/electrocardiogram  (EKG)  measure  has  a  reliability  of  0.60  for  CHF,  and  the 
three-item  medication  scale  for  CHF  has  a  reliability  of  0.97.  Scales  consisting  of  two  or 
fewer  items  generally  show  lower  reliabilities  even  though  they  may  make  clinical  sense  as  a 
scale.  For  example,  for  PNE,  the  day  3  MD  cognitive  assessment  process  has  only  two  items: 
MD  assess  shortness  of  breath  and  MD  examine  lungs.  Its  reliability  is  only  0.30.  In  certain 
cases,  we  cannot  even  estimate  a  reliability.  For  example,  for  PNE  day  3  nursing  care  there 
is  only  one  measurement:  RN  assess  shortness  of  breath.  Using  the  initial  nursing  items,  we 
might  guess  that  reliability  for  subsequent  RN  assessment  is  about  0.4. 


Table  C.50 


SOME   INITIAL   SCALE  RELIABILITIES  ACROSS  DISEASES 


Disease 


Initial  MD 

Cognitive 

Assessment 


Assessment 


Initial  RN 


Initial 
Laboratory- 
Assessment 


Congestive  heart 


0  .  83 


0.58 


0  .  87 


failure 
Acute  myocardial 


0.  85 


0.71 


0.93 


infarction 
Hip  Fracture 
Pneumonia 


0  .  68 
0  .71 


0.82 
0.  63 


0.90 
0.84 
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We  measure  only  a  few  of  the  multitude  of  processes  that  occur  during  the  hospital  stay. 
How  do  we  know  if  the  processes  we  measure  are  representative  of  the  concept  of  all  inhospi- 
tal  processes?  If  the  process  measures  correlate  with  each  other,  they  are  likely  to  be  mea- 
suring similar  concepts.  If  they  are  uncorrelated,  then  they  are  measuring  different  con- 
cepts. 

Using  reliabilities  for  PNE  patients  listed  in  Table  C.50,  we  can  calculate  the  expected 
intercorrelation  for  initial  MD  cognitive  assessment  (reliability  =  0.71)  and  initial  RN  assess- 
ment (reliability  =  0.63).  We  would  expect  a  correlation  of  (0.71)  (0.63)  =  0.45  if  the  two  process 
scales  were  different  measures  of  the  same  concept.  In  fact,  we  observed  only  0.16  correlation 
suggesting  initial  MD  cognitive  assessment  and  initial  RN  assessment  measure  different  pro- 
cess concepts. 

In  contrast,  initial  MD  cognitive  assessment  scales  correlate  well  with  subsequent  MD 
cognitive  assessment  scales.  For  example,  for  PNE,  the  expected  correlation  of  initial  MD 
cognitive  assessment  and  subsequent  MD  cognitive  assessment,  (0.71)  x  (0.30)  =  0.21,  is 
lower  than  the  observed  intercorrelation  of  0.39.  The  same  is  true  when  comparing  initial 
and  subsequent  RN  scales.  The  expectation  using  initial  RN  reliability  x  subsequent  RN  reli- 
ability, (0.63)  x  (approximately  0.4)  =  0.24,  is  less  than  the  observed  intercorrelation  of  0.30. 

Table  C.51  lists  observed  intercorrelations  for  PNE.  There  appears  to  be  clinical  valid- 
ity to  the  intercorrelation  values.  The  higher  intercorrelations  occur  where  we  would  expect, 
where  processes  are  more  likely  to  be  measuring  the  same  concept.  For  example,  initial  MD 
cognitive  and  RN  scales  correlate  well  with  subsequent  MD  and  RN  scales  (0.39  and  0.30, 


Table  C.51 


INTERCORRELATIONS  AMONG  THE  PROCESS  MEASURES 
FOR  PNEUMONIA 


Process 
Measure  1 


Process 
Measure  2 


Correlation  Between 
Process  Measures 
1  and  2 


MD  initial 
RN  initial 


MD  day  3 
RN  day  3 


0.39 
0.30 


MD  initial 
MD  day  3 


RN  initial 
RN  day  3 


0.16 
0.06 


MD  initial 
MD  initial 
Laboratory  studies 


Cultures 

Laboratory  studies 
Cultures 


0.22 
0.23 
0.17 


RN  initial 
RN  initial 


Cultures 

Laboratory  studies 


0.  07 
0.  07 


MD  initial 
RN  initial 


Time  to  first  antibiotic 
Time  to  first  antibiotic 


0.08 
0.  05 
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respectively).  MD  cognitive  assessment  has  low  intercorrelations  with  RN  assessment  on 
day  1  (0.16)  and  on  day  3  (0.06).  Since  the  physician  orders  cultures  (for  fever  work-up)  and 
laboratory  studies,  we  would  expect  adequate  intercorrelations  between  MD  processes  and 
ordering  laboratory  studies  and  cultures.  These  do  show  intercorrelations  (0.17-0.22)  con- 
sistent with  the  process  scales  measuring  similar,  but  not  identical  concepts.  Clinically,  the 
hospital-based  nurse  has  little  to  do  with  culture  and  laboratory  ordering.  The  intercorrela- 
tions (0.07)  support  the  view  that  nursing  process  measures  are  assessing  a  different  kind  of 
process  than  those  measuring  the  ordering  of  laboratory  studies  and  cultures. 

Interestingly,  there  is  a  very  low  intercorrelation  between  both  initial  MD  cognitive  and 
RN  initial  assessment  scales  and  the  amount  of  time  it  took  for  patients  to  receive  their  first 
antibiotic  dose.  Though  from  a  clinical  view  we  might  expect  these  to  correlate,  they  do  not, 
and  are  best  studied  as  separate  process  scales. 

We  used  these  principles  to  develop  a  series  of  process  scales  that  are  in  Tables  C.52 
through  C.55. 

Table  C.52 


PERFORMANCE  ON    INITIAL  PROCESS  SCALES 

FOR  5  DISEASES 

\  ri  c"    dllU     rUb[     r  ro 

Sample  Comb 

i  ned  ) 

Mean  %  Actual/Appl 

i  cab  1 e  Po  i  nts 

CHF 

AM  1 

PNE 

CVA 

H  1  P 

Initial   Process  Scales  (n=2824; 

1  (n=2853) 

(n=2749 

)     (n=2824)  | 

n=2762) 

MD  Cognitive  Assessment  74 

79 

64 

77 

80 

RN  Assessment  82 

81 

78 

67 

84 

Chest  X-ray  and  EKG  92 

92 

88 

82 

92 

Labs  87 

83 

85 

86 

88 

ICU  &  Telemetry  60 

86 

22 

39 

Med  i  ca  t  i  ons  & 

Special  Procedures 

02/ i ntuba t i on 

87 

02/ i ntuba t i on,   digitalis  94 

Medications  for  shock, 

CHF,   a rrthythm i cs. 

chest  pa  i  n 

65 

Swan-Ganz  catheter 

35 

Pacemaker 

66 

Streptokinase 

52 

Cu  1  tu res 

67 

Ant  i  b  i  ot  i  cs 

59 

Diagnostic  Tests  with  coma 

47 

Overall    Initial   Process  82 

82 

69 

76 

85 
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Table  C.53 

PERFORMANCE  ON  DAY  3   PROCESS  SCALES 
(Pre  and  Post  PPS  sample  combined) 


Mean  %  Actual/Appl i cable  Points 


CHF  AMI  PNE  CVA 

Day  3  Process  Scales  (n=2725)       (n=2545)       (n=2646)  (n=2679) 


MD  Cognitive  Assessment 

57 

66 

43 

67 

RN  Assessment 

88 

81 

78 

65 

Chest  X-ray  and  EKG 

37 

77 

54 

La  bs 

50 

54 

42 

47 

ICU  and  Telemetry 

61 

88 

33 

40 

Use  of  Medications 

92 

64 

92 

Overall   Day  3  Process 

73 

76 

51 

64 

COMBINING  PROCESS  POINTS  FOR  PROCESS  SCALE  SCORES 

For  each  patient,  we  summed  the  assigned  and  applicable  points  for  all  process  criteria. 
If  a  process  criterion  was  not  applicable,  we  added  zero  to  the  sum  of  assigned  points  and 
zero  to  the  sum  of  applicable  points.  If  a  criterion  was  applicable  but  was  not  met,  we  added 
zero  to  the  sum  of  assigned  points  and  we  added  the  full  number  of  applicable  points  to  the 
sum  of  applicable  points.  In  this  latter  instance,  the  sum  of  assigned  points  and  the  sum  of 
applicable  points  include  a  measure  of  poor  process. 

Next  we  determined  the  ratio  of  the  sum  of  assigned  points  to  the  sum  of  applicable 
points  for  each  patient.  We  used  a  mean  of  the  ratios  for  all  patients  to  combine  process  data 
across  patients.  These  means  are  referred  to  as  "mean  percent  of  assigned/applicable  points." 

For  example,  consider  the  process  criteria  for  monitoring  with  an  ICU  described  in 
Table  C.8.  Assume  that  the  patient  is  not  very  sick  on  day  1  or  day  2.  The  ICU  is  not  appli- 
cable for  this  patient.  We  score  this  patient  zero  assigned  points  and  zero  applicable  points 
for  each  day.  Since  the  patient  has  no  applicable  points  for  ICU,  we  do  not  include  this 
patient  in  the  summary  measure  combining  data  across  patients  (i.e.,  the  mean  percent  of 
assigned/applicable  points)  for  the  ICU. 

Alternatively,  assume  this  patient  was  very  sick  on  day  1  and  day  2,  but  was  only  in  the 
ICU  on  day  2  and  had  no  monitoring  with  ICU  or  telemetry  on  day  1.  We  score  this  patient  7 
assigned  and  7  applicable  points  on  day  2,  and  zero  assigned  and  7  applicable  points  on  day 
1.  The  ratio  of  assigned/applicable  points  for  ICU  for  this  patient  would  be  7/14  or  0.5.  We 
include  this  patient's  ratio  of  0.5  in  the  "mean  percent  of  assigned/applicable  points." 

If  the  patient  was  very  sick  on  day  1  and  not  very  sick  on  day  2  and  had  no  monitoring  by 
ICU  or  telemetry  on  either  day,  we  score  this  patient  zero  assigned  and  7  applicable  points  on 
day  1  and  zero  assigned  and  zero  applicable  points  on  day  2.  The  ratio  of  assigned/applicable 
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points  for  ICU  for  this  patient  would  be  0/7  or  0.  We  do  include  this  patient's  ratio  in  the  "mean 
percent  of  assigned/applicable  points." 

PERFORMANCE  ON  PROCESS  SCALES 

We  used  the  principles  of  scale  development  and  the  process  measures  described  pre- 
viously to  develop  six  initial  process  scales  measuring  process  on  the  first  two  hospital  days 
(Table  C.52  for  the  pre-  and  post-PPS  sample  combined).  Table  C.53  shows  six  scales  and  an 
overall  scale  measuring  day  3  process  for  the  pre-  and  post-PPS  sample  combined  as  a  proxy 
for  subsequent  hospital  days.  Table  C.54  compares  performance  on  initial  process  scales  for 
the  pre-  versus  post-PPS  sample.  Table  C.55  compares  performance  on  day  3  process  scales 
for  pre-  versus  post-PPS  sample. 


Appendix  D 


FURTHER  DETAILS  ON  MEASURING  INSTABILITY 

AT  DISCHARGE 


Tables  D.l  through  D.5  show  the  regressions  of  instability  at  discharge  (a  dichotomous 
variable)  on  overall  process  of  care  and  sickness  at  admission  for  each  disease.  The  sample 
excludes  patients  with  inhospital  death,  do  not  resuscitate  orders,  or  transfers  to  acute  care 
hospitals. 

Tables  D.6  through  D.9  show  regressions  as  in  Tables  D.l  through  D.5.  However,  in 
Tables  D.6  through  D.7,  the  sample  excludes  patients  with  a  preadmission  residence  of  nurs- 
ing home,  in  addition  to  the  exclusions  noted  in  Tables  D.l  through  D.5. 

Table  D.10  shows  the  percentage  of  patients  with  measures  of  instability  present  at 
discharge  in  the  pre-  compared  with  the  post-PPS  period.  The  study  sample  excludes  inhos- 
pital deaths,  transfers  to  acute  care  hospitals,  and  patients  with  "do  not  resuscitate"  orders 
written  ever  during  the  hospital  stay. 

Table  D.ll  shows  the  percentage  of  patients  with  measures  of  sickness  at  discharge  in 
the  pre-  compared  with  the  post-PPS  period.  The  study  sample  excludes  inhospital  deaths, 
transfers  to  acute  care  hospitals,  and  patients  with  "do  not  resuscitate"  orders  written  ever 
during  the  hospital  stay. 


302 


303 


Table  D.l 


REGRESSION  OF  INSTABILITY  AT  DISCHARGE 
ON  PROCESS  AND  SICKNESS  AT  ADMISSION  FOR 
PATIENTS  WITH  CONGESTIVE  HEART  FAILURE 
FOR  SAMPLE  PATIENTS 


Source  |  SS  df  MS  Number  of  obs  -  2342(4) 
 +   F(     3,      2338)  =         3  .89 

Model  |  2.62210975  3  .874036583  Prob  >  F  =  0.0088 
Residual  |  525.160982  2338  .224619753  R-square  =  0.0050 
 +   Adj  R-square  =  0,  0037 

Total   |     527.783091     2341     .225451983  Root  MSE  =  .47394 

Variable   |     Coefficient        Std.  Error  t        Prob  >   |t|  Mean 

 +  ■  

dinstab  |  .1827498 

 +  

process(l|)        -.0252842             .0105609         -2.394           0.017  .0641805 
sev30(2|)           .0062882              .0031051           2.025           0.043  31.00482 
sevl80(3|)         -.0027089              .0021021         -1.289            0.198  36.28997 
_COns    I            .0877145              -0434954           2.017           0.044  1 
 +  


(1)  Overall  process  scale.     See  Sec  IV.   for  details. 

(2)  Sev  30  is  the  sickness  at  admission  scale  designed  to  predict  death 
within  30  days  post-admission.     See  Sec.   Ill  for  details. 

(3)  Sev  180  is  the  sickness  at  admission  scale  designed  to  predict 
death  within  180  days  post-admission.     See  Sec.   Ill  for  details. 

(4)  Includes  patients  admitted  from  home  or  from  a  nursing  home. 
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Table  D.2 


REGRESSION  OF  INSTABILITY  AT  DISCHARGE 
ON  PROCESS  AND  SICKNESS  AT  ADMISSION  FOR 
PATIENTS  WITH  ACUTE  MYOCARDIAL  INFARCTION 
FOR  SAMPLE  PATIENTS 


Source  | 

SS 

df 

MS 

Number  of  obs 
F(     3,  1942) 
Prob  >  F 
R-square 
Adj  R-square 
Root  MSE 

194  6( 
1.90 
=  0.1250 
=  0.0029 
=  0.0014 
=  .32129 

Model  | 
Residual  j 

.589727516 
200.467313 

3 

1942 

.196575839 
.103227246 

Total  | 

201. 05704 

1945 

. 103371229 

Variable  | 

Coefficient 

Std 

.  Error 

t 

Prob  >  |t| 

Mean 

 +- 

dinstab  | 

. 1012333 

 +- 

process(  1|) 
sev3  0(2|) 
sevl80(  3)) 
_cons  | 

-.0127985 
-.0013617 
-.0000462 
.1334108 

• 

.0082 
0015676 
. 00108 
0191714 

-1 
-0 
-0 
6 

.561 
.869 
.  043 
.959 

0.119 
0.385 
0.966 
0.000 

.1061341 
21.45644 
34 . 65304 
1 

+ 


(1)  Overall  process  scale.     See  Sec  IV.   for  details. 

(2)  Sev  30  is  the  sickness  at  admission  scale  designed  to  predict  death 
within  30  days  post-admission.     See  Sec.   Ill  for  details. 

(3)  Sev  180  is  the  sickness  at  admission  scale  designed  to  predict 
death  within  180  days  post-admission.     See  Sec.   Ill  for  details. 

(4)  Includes  patients  admitted  from  home  or  from  a  nursing  home. 
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Table  D.3 

REGRESSION  OF  INSTABILITY  AT  DISCHARGE 
ON  PROCESS  AND  SICKNESS  AT  ADMISSION  FOR 
PATIENTS  WITH  PNEUMONIA 
FOR  SAMPLE  PATIENTS 


Source  |  SS  df  MS  Number  of  obs  =  2143(4) 
 +   F(      3,      2139)  =  9.25 

Model  |  4.82600703  3  1.60866901  Prob  >  F  =  0.0000 
Residual  j  371.86368  2139  .173849313  R-square  =  0.0128 
 +   ^dj  R-square  =  0.0114 

Total    |      376.689687      2142      .175858864  RootMSE  =  .41695 

Variable  |     Coefficient        Std.  Error  t        Prob  >  |t|  Mean 

 +  

dinstab   |  .1577228 

 +  

process(l|)         -.0059021                .009281         -0.636           0.525  .014871 
sev30(2|)           .0015728                  .00236           0.666           0.505  22.55418 
sevl80(3|)           .0016339              .0018999           0.860           0.390  27.97711 
_cons    j            .0766255              .0179909            4.259            0.000  1 
 +  


(1)  Overall  process  scale.     See  Sec  IV.   for  details. 

(2)  Sev  30  is  the  sickness  at  admission  scale  designed  to  predict  death 
within  30  days  post-admission.     See  Sec.   Ill  for  details. 

(3)  Sev  180  is  the  sickness  at  admission  scale  designed  to  predict 
death  within  180  days  post-admission.     See  Sec.   Ill  for  details. 

(4)  Includes  patients  admitted  from  home  or  from  a  nursing  home. 
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Table  D.4 


REGRESSION  OF  INSTABILITY  AT  DISCHARGE 
ON  PROCESS  AND  SICKNESS  AT  ADMISSION  FOR 
PATIENTS  WITH  CEREBROVASCULAR  ACCIDENT 
FOR  SAMPLE  PATIENTS 


Source  ! 

SS 

df 

MS 

Number   of  obs 

1927  C 

 1 — 

F (     3 ,      1 923 ) 

12.29 

Model  ! 

8. 55950834 

2.85316945 

Prob   >  F 

=     0 . 0000 

Residual  ! 

446. 585276 

1923 

. 232233633 

R— square 

=  0.0188 



Ad j   R— square 

=  0.0173 

Total  ! 

455. 144785 

1926 

. 236316088 

Root  MSE 

=  .48191 

Variable  ! 

Coef  -f  i  c  i  ent 

Std 

.  Error 

t 

Prob    >    ! t I 

Mean 

dinstab  ! 

.2376751 

process  (*) 

-.0312874 

01 18497 

-2. 

640 

0 .  008 

. 0352452 

-.0008432 

0028251 

-0. 

300 

0.  764 

14. 90474 

sevlBO'V 

. 0057283 

0022188 

<-> 

582 

0.01 0 

23.58632 

_cons  ! 

. 1 1 6309 

0254959 

4. 

562 

0 .  000 

1 

<0 


(1)  Overall  process  scale.     See  Sec  IV.   for  details. 

(2)  Sev  30  is  the  sickness  at  admission  scale  designed  to  predict  death 
within  30  days  post-admission.     See  Sec.   Ill  for  details. 

(3)  Sev  180  is  the  sickness  at  admission  scale  designed  to  predict 
death  within  180  days  post-admission.     See  Sec.   Ill  for  details. 

(4)  Includes  patients  admitted  from  home  or  from  a  nursing  home. 
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Table  D.5 


REGRESSION  OF  INSTABILITY  AT  DISCHARGE 
ON  PROCESS  AND  SICKNESS  AT  ADMISSION  FOR 
PATIENTS  WITH  HIP  FRACTURE 
FOR  SAMPLE  PATIENTS 


Source  |  SS  df  MS  Number  of  obs  =  2551(4) 
 +   F(     3,      2547)    =  21.70 

Model  |  15.3375361  3  5.11251204  Prob  >  F  =  0.0000 
Residual  j  600.179516  2547  .235641742  R-square  =  0.0249 
 +   Adj  R-square     =  0.0238 

Total   |     615.517052     2550      .241379236  Root  MSE             =  .48543 

Variable  |     Coefficient        Std.   Error             t        Prob  >   |t|  Mean 
 +  

dinstab   |  .2371619 
 +  

process(]))        -.0027941              .0095015         -0.294            0.769  -.0033951 
sev30(2,)          .0020803             .0030464           0.683           0.495  6.998122 
sevl80(3,)           .0069131          ^.0014888           4.  643            0.000              14  .24408 
_cons    j            .1241237              .0174399           7.117            0.000  1 
 +  


(1)  Overall  process  scale.     See  Sec  IV.   for  details. 

(2)  Sev  30  is  the  sickness  at  admission  scale  designed  to  predict  death 
within  30  days  post-admission.     See  Sec.   Ill  for  details. 

(3)  Sev  180  is  the  sickness  at  admission  scale  designed  to  predict 
death  within  180  days  post-admission.     See  Sec.   Ill  for  details. 

(4)  Includes  patients  admitted  from  home  or  from  a  nursing  home. 
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Table  D.6 


REGRESSION  OF  INSTABILITY  AT  DISCHARGE 
ON  PROCESS  AND  SICKNESS  AT  ADMISSION  FOR 
PATIENTS  WITH  CONGESTIVE  HEART  FAILURE 
FOR  PATIENTS  ADMITTED  FROM  HOME 


Source 


SS 


df 


MS 


Model  |  3.48699844  3  1.16233281 
Residual  |  439.580731  2004  .219351662 
 +  

Total    |      443.067729     2007  .2207612 


Number  of  obs  = 
F(  3,  2004)  = 
Prob  >  F  = 
R-square  = 
Adj  R-square  = 
Root  MSE 


2008(4) 

5.30 
0.0014 
0.0079 
0.0064 
.46835 


Variable  | 
 +- 

Coefficient 

Std.  Error 

t 

Prob 

>  |t| 

Mean 

dinstab  | 

. 1713147 

 +- 

process(  1|) 

-.0314093 

. 0116404 

-2 

.  698 

0 . 

007 

. 1144944 

sev30(3) 

. 0090492 

. 0033602 

2 

.  693 

0. 

007 

30. 65672 

sevl80(3|) 

-.0044498 

. 0022858 

-1 

.  947 

0. 

052 

35.53591 

cons  j 
 +- 

. 0556204 

.0474292 

1 

.173 

0. 

241 

1 

(1)  Overall  process  scale.     See  Sec  IV.   for  details. 

(2)  Sev  30  is  the  sickness  at  admission  scale  designed  to  predict  death 
within  30  days  post -admiss ion .     See  Sec.   Ill  for  details. 

(3)  Sev  180  is  the  sickness  at  admission  scale  designed  to  predict 
death  within  180  days  post-admission.     See  Sec.   Ill  for  details. 

(4)  Includes  patients  admitted  from  home  or  from  a  nursing  home. 
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Table  D.7 


REGRESSION  OF  INSTABILITY  AT  DISCHARGE 
ON  PROCESS  AND  SICKNESS  AT  ADMISSION  FOR 
PATIENTS  WITH  ACUTE  MYOCARDIAL  INFARCTION 
FOR  PATIENTS  ADMITTED  FROM  HOME 


Source  |  SS  df  MS  Number  of  obs  =  17  3  7(4) 
 +   F(     3,      1733)   =  2.10 

Model  |  .652496326  3  .217498775  Prob  >  F  =  0.0973 
Residual  |  179.901332  1733  .103809193  R-square  =  0.0036 
 +   Adj  R-square     =  0.0019 

Total    |      180.553828     1736     .104005662  Root  MSE             =  .32219 

Variable  |     Coefficient        Std.  Error  t        Prob  >  |t|  Mean 

 +  

dinstab   |  .1030512 

 +  

process(J)        -.0179784              .0088459         -2.032            0.042  .14738 
sev30(3)        -.0014555              .0016814         -0.866           0.387  21.22408 
sevl80(3J           .0001663              .0011754            0.141           0.888  34.05825 
_cons    |            .1309308                    .0203            6.450           0.000  1 
 +  


(1)  Overall  process  scale.     See  Sec  IV.   for  details. 

(2)  Sev  30  is  the  sickness  at  admission  scale  designed  to  predict  death 
within  30  days  post-admission.     See  Sec.   Ill  for  details. 

(3)  Sev  180  is  the  sickness  at  admission  scale  designed  to  predict 
death  within  180  days  post-admission.     See  Sec.   Ill  for  details. 

(4)  Includes  patients  admitted  from  home  or  from  a  nursing  home. 
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Table  D.8 

REGRESSION  OF  INSTABILITY  AT  DISCHARGE 
ON  PROCESS  AND  SICKNESS  AT  ADMISSION  FOR 
PATIENTS  WITH  PNEUMONIA 
FOR  PATIENTS  ADMITTED  FROM  HOME 


Source  |  SS  df  MS  Number  of  obs  =  1623(4) 
 +   F(     3,      1619)  =  6.57 

Model  |  3.31009859  3  1.1033662  Prob  >  F  =  0.0003 
Residual  j  272.09594  1619  .1680642  R-square  =  0.0120 
 +—   Adj  R-square  =  0.0102 

Total    |      275.406038     1622      .169794105  Root  MSE  =  .40996 

Variable  |     Coefficient        Std.  Error             t        Prob  >   |t|  Mean 
 +  ■  

dinstab  |  .1417129 


processd)            .0036878              .0108757            0.339            0.735  .1281555 
sev30(2|)            .0025611              .0028124            0.911           0.363  20.03255 
sevl80(3|)              .000962              .0022616            0.425           0.671  24.69526 
_COns    |            .0661784              .0198653            3.331           0.001  1 
 +  ■  


(1)  Overall  process  scale.     See  Sec  IV.   for  details. 

(2)  Sev  30  is  the  sickness  at  admission  scale  designed  to  predict  death 
within  30  days  post-admission.     See  Sec.   Ill  for  details. 

(3)  Sev  180  is  the  sickness  at  admission  scale  designed  to  predict 
death  within  180  days  post-admission.     See  Sec.   Ill  for  details. 

(4)  Includes  patients  admitted  from  home  or  from  a  nursing  home. 
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Table  D.9 


REGRESSION  OF  INSTABILITY  AT  DISCHARGE 
ON  PROCESS  AND  SICKNESS  AT  ADMISSION  FOR 
PATIENTS  WITH  HIP  FRACTURE 
FOR  PATIENTS  ADMITTED  FROM  HOME 


Source  |  SS  df  MS  Number  of  obs  =        18  8  9(4) 

 +   p(     3,      1885)  -  17.90 

Model   |      12.1780955  3     4.05936517  Prob  >  F  =  0.0000 

Residual    |         427.3984     1885     .226736552  R-square  =  0.0277 

 +   Adj  R-square  =  0.02  62 

Total    |     439.576496     1888     .232826534  Root  MSE  =  .47617 

Variable   |     Coefficient        Std.  Error  t        Prob  >   |t|  Mean 

 +  

dinstab    |  .2175754 
 +  

process^)        -.0003077              .0107957         -0.029            0.977  -.0032458 
sev30<3)           .0028907              .0038577            0.749            0.454  6.724002 
sevl80(3>           .0074071              .0020696           3.579           0.000  11.61056 
_cons               .1121364             .0187481           5.981           0.000  1 
 +  


(1)  Overall  process  scale.     See  Sec  IV.   for  details. 

(2)  Sev  30  is  the  sickness  at  admission  scale  designed  to  predict  death 
within  30  days  post-admission.     See  Sec.   Ill  for  details. 

(3)  Sev  180  is  the  sickness  at  admission  scale  designed  to  predict 
death  within  180  days  post -admission .     See  Sec.   Ill  for  details. 

(4)  Includes  patients  admitted  from  home  or  from  a  nursing  home. 
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METHODS  FOR  STUDYING  OUTCOMES  SUBSEQUENT 
TO  ACUTE  CARE  HOSPITAL  DISCHARGE 


POSTDISCHARGE  MORTALITY:  DATA  CLEANING 

To  study  the  effect  of  DRG-based  prospective  payment  on  outcomes,  we  have  supple- 
mented the  study  of  inhospital  outcomes  with  the  study  of  postdischarge  death,  readmis- 
sions,  and  duration  of  stay  in  a  nursing  home. 

Using  the  Medical  Record  as  a  Data  Base 

We  began  the  process  of  accessing  the  data  by  collecting  data  from  the  medical  record  of 
all  cases  abstracted  regardless  of  whether  the  case  was  ultimately  included  in  or  excluded 
from  the  study  sample.  From  each  medical  record  we  sought  the  Health  Insurance  Claim 
(HIC)  number  (consisting  of  a  nine  digit  Social  Security  number  and  a  three  digit  Beneficiary 
Insurance  Claim  (BIC)  number).  If  no  HIC  number  was  available  from  the  medical  record, 
we  supplied  the  Social  Security  number  (SSN);  this  was  done  for  5  percent  of  the  cases.  The 
data  generated  from  the  medical  record  are  known  as  the  RAND  file. 

Using  HCFA  Files  as  the  Data  Base 

We  used  existing  HCFA  files  as  the  source  for  postdischarge  mortality  and  readmission 
data.  The  HCFA  file  provided  patients'  Health  Insurance  Claim  (HIC)  numbers,  last  and 
first  names,  and  dates  of  birth.  The  goal  of  this  effort  was  to  match  HIC  numbers  between 
the  RAND-generated  file  (based  upon  medical  record  data)  and  the  HCFA  files.  HCFA  used 
the  Health  Insurance  Master  File  (also  known  as  HIM  or  HIPO)  to  provide  mortality  data 
and  the  Bill  Retrieval  File  to  provide  readmission  data. 

Matching  the  RAND  Medical  Record  File  and  the  HCFA  Files 

Matching  files  by  HIC  number  will  have  limited  success  in  accessing  valid  mortality 
data  if  the  HIC  number  elicited  from  the  medical  record  (or  recorded  in  the  RAND  file)  is 
missing  or  inaccurate.  To  determine  the  number  of  cases  that  could  be  accurately  matched 
between  the  RAND  and  HCFA  files,  we  developed  an  extensive  data  cleaning  process.  Table 
E.l  diagrams  the  method  in  which  the  data  were  received  and  cleaned.  Initially,  we  submit- 
ted the  RAND  file  with  data  for  19,540  patients  (included  and  excluded  abstraction  forms)  to 
HCFA.  This  number  was  88  percent  of  the  total  number  of  cases  eventually  submitted  to 
HCFA.  We  submitted  only  88  percent  of  cases  at  this  time  because  fieldwork  was  incomplete. 

Matching  was  done  by  HCFA;  they  compared  RAND  and  HCFA  HIC  numbers.  HCFA 
returned  to  RAND  more  records  than  we  sent  (n  =  19,595),  since  HCFA  provided  some  dupli- 
cate matches  for  cases  that  had  missing  BICs.  We  were  able  to  use  HCFA's  Health 
Insurance  Master  File  to  find  mortality  information  for  87.2  percent  of  the  cases.  The 
remaining  12.8  percent  of  cases  (n  =  2,514)  could  be  categorized  as  follows:  no  match  because 
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Table  E.l 

MATCHING  THE   RAND  MEDICAL  RECORD   FILE  AND  THE  HCFA  FILES 

19,540  Include  &  exclude  records  sent  to  HCFA 
I 

19,595  Records  received  from  HCFA 


17, 081 
(87.2%) 
HIC  number  match 


2514 
(12.8%) 
No  match 


12,218 
(62.4%) 

correct  match  on 
last  name,  first 
name,   date  of  birth 


3639 
(18.6%) 
Questionable 
names,  dob 


1224 
(6.2%) 
Incorrect 
names,  dob 
I 


9,  465 

Include 

only 


records 


Sent  to  HCFA  to  obtain 
corrected  HIC  numbers  by 
matching  on  last  &   first  name 


I 

7,170 

Include  records  only. 

File  recieved  from 

HCFA  after  matching 

using  corrected  HIC  numbers 


16, 635 

Include  records 
received  from  HCFA  after 
matching  by  HIC  number  and- 
corrected  HIC  number 
I 


15,413 

92.65% 
Correctly 
matched  on 
last  name, 
first  name,  & 
date  of  birth. 
Includes  hand 
check 


34 
.29% 

Correctly 
matched  on 
last  name, 
first  name,  & 
date  of  birth, 
but  no  mortal- 
ity information 


932 
5.  60% 
Questionable 
patients 


 I 

I 

256 
1.54% 
Incorrectly 
matched  on 
last  name, 
first  name,  & 
date  of  birth 


16, 766 

Include  patients 
in  our  dataset 


15, 381 
92 . 00% 
Match  with 
RAND  dataset 


1,  399 
8  .  00% 

No  match  with 
RAND  dataset 
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the  patient's  medical  record  (and  so  the  RAND  file)  had  a  Railroad  Board  number  instead  of  a 
HIC  number  (2.1  percent,  n  =  402);  no  match  because  of  an  invalid  HIC  number  (4.8  percent, 
n  =  934);  no  match  because  neither  HIC  number  nor  SSN  appeared  on  the  medical  record  (2.7 
percent,  n  =  523);  and  finally,  HIC  number  was  identified  as  no  longer  current,  but  a  cross 
reference  number  was  provided  that  could  be  used  in  subsequent  searches  (3.4  percent,  n  = 
655). 

For  patients  who  had  a  HIC  number  match  (87.2  percent),  RAND  verified  that  the  infor- 
mation provided  on  the  file  actually  corresponded  to  RAND's  study  patients.  The  verification 
was  necessary,  since  it  was  possible  for  a  HIC  number  to  be  valid  but  not  belong  to  a  study 
patient.  This  was  a  problem  particularly  when  the  patient  was  not  the  primary  wage  earner, 
and  thus  used  his  or  her  spouse's  Social  Security  number.  To  verify  the  accuracy  of  the  HIC 
numbers  of  the  87.2  percent  of  patients,  we  compared  first  name,  last  name,  and  date  of  birth 
between  the  RAND  medical  record  data  and  the  data  obtained  from  the  HIM  (HIPO)  file. 

Roughly  62  percent  of  the  cases  received  from  HCFA  matched  exactly  on  all  three  vari- 
ables and  were  considered  "correct."  An  additional  19  percent  matched  on  one  or  two  vari- 
ables, and  were  classified  as  "questionable."  The  remaining  6  percent  did  not  match  on  first 
name,  last  name,  or  date  of  birth,  and  were  considered  "incorrect."  Because  these  last  two 
categories  plus  the  12.9  percent  with  no  matches  constituted  over  one-third  of  the  data,  we 
used  another  source  to  check  the  validity  of  the  HIC  numbers  for  the  cases  without  exact 
matches. 

We  supplemented  the  HIC  number  matches  with  a  second  analysis  of  HCFA  files.  The 
purpose  of  this  second  analysis  was  to  match  RAND  patients  with  HCFA  patients  using  last 
and  first  names.  This  allowed  us  to  verify  HIC  numbers  and  to  correct  HIC  numbers  for 
name  matches  if  the  HIC  number  deviated  slightly.  The  newly  corrected  HIC  numbers  were 
forwarded  again  to  HCFA  for  a  final  run  through  the  Health  Insurance  Master  File  and  the 
Bill  Retrieval  File. 

We  used  one  other  method  for  accessing  HIC  numbers.  When  the  medical  record  pro- 
vided no  data  about  HIC  numbers,  we  worked  with  state-specific  insurance  carriers  to  access 
those  data.  They  were  able  to  provide  HIC  numbers  for  roughly  60-70  percent  of  patients 
whose  medical  record  provided  no  data  about  HIC  numbers.  These  HIC  numbers  were  also 
submitted  to  HCFA  to  be  run  through  the  Health  Insurance  Master  File  and  Bill  Retrieval 
File. 

A  check  for  accuracy  was  performed  on  this  file,  which  included  a  "hand  check"  of 
records  that  did  not  match  exactly  on  all  three  of  the  variables  listed  above.  Variables  are 
considered  correctly  matched  if  there  are  obvious  misspellings  or  transpositions.  In  general, 
if  at  least  two  of  the  three  variables  matched,  whether  by  exact  match  or  "hand  check,"  the 
record  was  considered  correct. 

Based  on  an  exact  match,  70  percent  of  the  included  records  were  correct.  The 
remainder  of  the  cases  were  considered  questionable.  Using  the  rules  listed  in  Table  E.2, 
another  22  percent  of  the  records  were  considered  correct  by  the  "hand  check"  criteria. 

The  most  frequent  source  of  patient  mismatch  (last  name,  first  name,  date  of  birth) 
when  HIC  numbers  matched  between  RAND  and  HCFA  files  was  matching  error,  because 
patients  used  spouses'  HIC  numbers.  This  was  a  problem  for  up  to  1.2  percent  of  the 
included  cases.  All  other  matching  errors  combined  made  up  fewer  than  1  percent.  HCFA 
was  unable  to  match  HMO  patients  as  well  as  undocumented  patients  who  still  had  unclean 
HIC  numbers,  and  subsequently,  4.3  percent  of  the  included  cases  were  not  matched.  In 
summary,  for  patients  on  whom  we  had  HIC  numbers  on  both  the  RAND  and  HCFA  file,  we 
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Table  E.2 

CRITERIA  FOR  DECIDING  WHETHER  QUESTIONABLES  ARE  CORRECT 


Last        First      Date  of 

Name        Name  Birth  Keep  if: 

1.  -  +  a.     First  and  last  names  reversed. 

b.     First  and  last  names  obviously  correct 
but  misspelled. 

2.  -  +  +  a.     Last  names  obviously  correct  but 

misspelled, 
b.     Different  last  names  and  female  first 
names  since  these  are  probably  women 
using  their  maiden  names. 

3.  -  +  -  a.     Last  names  obviously  correct  but 

misspelled  or  patient  is  female  (using 
maiden  name)  and  date  of  birth  not  off 
by  more  than  5  years, 
b.     Last  names  obviously  correct  but 

misspelled  or  patient  is  female  (using 
maiden  name)  and  month,  day,  or  year  of 
birth  interchanged. 

4.  +  -  +  a.     First  names  obviously  correct  but 

misspelled . 

b.  HCFA  first  initials  match  the  first 
character  of  RAND  first  names. 

c.  First  name  and  middle  name  reversed. 

5.  +  -  -  a.     First  names  obviously  correct  but 

misspelled  and  date  of  birth  not  off 
by  more  than  5  years. 

b.  First  names  obviously  correct  but 
misspelled  and  month,  day,  or  year  of 
birth  interchanged. 

c.  HCFA  first  initials  match  the  first 
character  of  RAND  first  names  and 

date  of  birth  not  off  by  more  than  5  years 

d.  HCFA  first  initials  match  the  first 
character  of  RAND  first  names  and  month, 
day,  or  year  of  birth  interchanged. 

— >  Incorrect  individuals  in  this  category 

were  usually  wives  using  their  husband's 
HIC  numbers. 

6.  +  +  -  a.     All  considered  correct. 
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found  92.65  percent  of  patients  had  an  acceptable  match  on  last  name,  first  name,  and  date  of 
birth.  This  file  is  92.65  percent  correct.  (The  figure  of  62  percent  cited  earlier  was  relevant 
to  both  included  and  excluded  sample  patients  and  the  92.65  percent  is  relevant  to  included 
sample  patients  only.  A  more  thorough  verification  check  was  used  for  the  included  sample.) 

The  next  step  was  to  link  the  92.65  percent  (n  =  15,413)  patients  for  whom  we  had  clean 
mortality  information  with  the  HCFA  databases.  We  were  able  to  obtain  a  match  on  92  per- 
cent of  the  cases.  The  remaining  8  percent  consist  of  patients  for  whom  HCFA  was  unable  to 
match  for  reasons  specified  above.  Table  E.3  presents  the  pre/post  PPS  breakdown  of  the  8 
percent  who  do  not  match. 

A  higher  proportion  of  pre-PPS  (61.3  percent)  than  post-PPS  (38.7  percent)  patients  did 
not  match.  This  may  be  due  to  data  abstracting  errors.  Records  that  are  six  years  old  are 
more  faded  and  therefore  harder  to  read  than  records  that  are  two  years  old.  This  increases 
the  likelihood  that  a  HIC  number  will  be  misread  and  subsequently  cause  a  nonmatching 
record. 

Nonmatching  patients  who  died  inhospital  are  similarly  distributed  with  62.9  percent 
pre-,  and  37.1  percent  post-PPS  (Table  E.4).  Since  inhospital  death  does  not  differ  with  study 
period  for  patients  who  died  inhospital  compared  with  all  nonmatching  patients,  inhospital 
deaths  were  not  disproportionately  represented  among  the  nonmatching  patients.  We  found 
that  patients  with  inaccurate  data  who  died  inhospital  are  distributed  similarly  across  time 
periods:  20  percent  pre-PPS  compared  with  19  percent  post-PPS  (p  a  0.05). 

Table  E.3 

ABSTRACT  FORM  INCLUDES  CASES  WITHOUT  ADEQUATE  MATCH 
IN  POSTDISCHARGE  FILE  BY  STUDY  PERIOD 


Cumulative 

Cumulative 

Period 

Frequency 

Percent 

Frequency 

Percent 

Pre 

858 

61.3 

858 

61.3 

Post 

541 

38.7 

1399 

100.0 

Table  E.4 

ABSTRACT   FORM  INCLUDES  CASES  WITHOUT  ADEQUATE  MATCH 
IN  POSTDISCHARGE  FILE  BY  STUDY  PERIOD: 
IN-HOSPITAL  DEATHS  ONLY 


Cumulative 

Cumulative 

Period 

Frequency 

Percent 

Frequency 

Percent 

Pre 

173 

62.9 

173 

62.9 

Post 

102 

37  . 1 

275 

100.0 

319 


We  studied  this  further  by  evaluating  the  accuracy  of  the  date  of  death  as  reported  in 
both  HCFA's  HIM  file  and  the  RAND  medical  record  file.  Table  E.5  shows  the  percentage  of 
cases  with  agreement  in  date  of  death  for  each  of  the  study  years.  Table  E.6  shows  the  per- 
centage of  cases  with  agreement  in  date  of  death  for  the  pre-PPS  versus  post-PPS  periods. 
Both  Tables  E.5  and  E.6  show  a  very  high  fraction  of  cases  (>  94  percent)  with  agreement  in 
date  of  death  between  the  two  data  files. 

For  the  purpose  of  analysis,  patients  who  died  after  March  23,  1988,  were  censored  at 
that  date.  This  was  the  date  that  the  initial  postdischarge  mortality  file  was  received,  and 
since  only  incorrect  records  were  sent  back  to  HCFA  to  be  cleaned,  mortality  information  on 
correct  records  was  not  available  after  this  date.  Another  correction  was  made  if  the  patient 
died  inhospital  and  the  postdischarge  mortality  date  was  different  from  the  discharge  date: 
Then,  the  date  of  death  was  taken  as  the  hospital  discharge  date,  because  there  is  sometimes 
a  lag  in  reporting  death  dates,  and  hospital  discharge  dates  were  considered  more  accurate. 

Table  E.5 


VLKll  1LA1  J.UJN  Ur 

AND  HCFA 

DATA  SETS 

lINnUb.Fl  lAij 
BY  STUDY 

YEAR   ( 1 ) 

KANL) 

Study  Year 

Dates  of  Inhospital 
Death  Agree  Within: 

1981 
(n=362) 

1982 
(n=537) 

1985/86 
(n=851 ) 

Total 
(n=1750) 

1  Day 

19.4 

28.6 

45.8 

93.8 

>  1  day  -  <  1  week 

0.6 

0.9 

1.1 

2.6 

>  2  weeks  -  3  weeks 

0.3 

0.3 

0.7 

1.3 

>  3  weeks  -  4  weeks 

0.2 

0.3 

0.3 

0.9 

>  4  weeks 

0.1 

0.2 

0.4 

0.7 

>  2  -  3  months 

0.0 

0.2 

0.2 

0.4 

>  3  -  4  months 

0.0 

0.0 

0.1 

0.1 

3-4  months 

0.0 

0.1 

0.0 

0.1 

>  4  months 

0.2 

0.1 

0.0 

0.2 

Total 

20  .  7 

30.7 

48.6 

100.0 

(1)     This  appendix  includes  only  sample  patients  included 
in  the  final  study  sample  who  also  had  an  adequate  match  be- 
tween RAND  and  HCFA  files  who  also  died  inhospital. 
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A  final  correction  was  made  for  the  Record  Identification  Code  (RIC)  =  C  (cross  reference) 
patients.  Mortality  dates  for  these  patients  are  actually  the  dates  the  HIC  numbers  were 
changed.  These  patients  were  included  only  if  this  date  was  later  than  their  discharge  date 
and  were  censored  as  of  that  date. 

ADJUSTING  DISCHARGE  DESTINATION  FOR  SICKNESS  AT  ADMISSION 

Just  as  comparisons  of  rates  of  mortality  pre-  and  post-PPS  need  to  be  adjusted  for  sick- 
ness at  admission,  comparisons  of  rates  of  discharge  to  home  or  nursing  home  also  need  to  be 
adjusted  for  sickness  at  admission.  To  accomplish  this,  we  used  methods  similar  to  those 
used  to  develop  the  measures  designed  to  predict  death  (see  Sec.  III).  We  began  with  the 
same  variables  used  in  the  development  of  the  death  scales.  Each  variable  was  scored  by 
physicians  using  clinical  judgment  about  whether  the  variable  was  likely  to  predict  discharge 

Table  E.6 

VERIFICATION  OF  DATE  OF  INHOSPITAL  DEATH 


BY  RAND  AND 
BY  PPS 

HCFA  DATA 
PERIOD  (1) 

SETS 

Study  Period 

Dates  of  Inhospital 
Death  Agree  Within: 

Pre-PPS 
(n=899) 

Post-PPS 
(n=851 ) 

Total 
(n=1750) 

1  day 

48.0 

45.8 

93.8 

>  1  day  -  <  1  week 

1.5 

1.1 

2.6 

>  2  weeks  -  3  weeks 

0.6 

0.7 

1.3 

>  3  weeks  -  4  weeks 

0.5 

0.3 

0.9 

>  4  weeks 

0.3 

0.4 

0.7 

>  2  -  3  months 

0.2 

0.2 

0.4 

>  3  -  4  months 

0.0 

0.1 

0.1 

3-4  months 

0.1 

0.0 

0.1 

>  4  months 

0.2 

0.0 

0.2 

Total 

30.7 

48.6 

100.0 

(1)     This  appendix  includes  only  sample  patients 
included  in  the  final  study  sample  who  also  had  an 
adequate  match  between  RAND  and  HCFA  files  who  also 
died  inhospital. 
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to  home  or  a  nursing  home.  If  clinicians  believed  the  variable  was  likely  to  predict  discharge 
to  a  nursing  home  (score  >  0  using  a  0-4  rating  system)  and  if  the  variable  was  not  already 
included  in  the  disease-specific  scale  designed  to  predict  death,  then  the  variable  was  eligible 
for  inclusion  in  the  scale  to  predict  discharge  to  a  home  (rather  than  an  institution).  We  used 
logistic  regression  of  discharge  to  home  on  both  the  scale  to  predict  death  (Sec.  Ill),  and  the 
other  variables  rated  by  clinicians  to  predict  discharge  to  home  (rather  than  an  institution), 
to  develop  the  scale  to  predict  discharge  to  home.  Table  E.7  shows  this  regression  for 
patients  with  cerebrovascular  accident  as  an  example  of  the  methods  used.  Table  E.8  shows 
rates  of  discharge  to  nursing  homes  with  and  without  adjustment  for  sickness  at  admission 
using  the  scales  described  above  for  cerebrovascular  accident  patients. 

COMPARING  OUTCOMES  BEFORE  AND  AFTER  PPS  IMPLEMENTATION 

Table  E.9  presents  data  for  mean  length  of  stay  pre-  and  post-PPS  after  adjustment  for 
sickness  at  admission.  These  adjusted  length-of-stay  values  do  not  differ  substantially  from 
the  unadjusted  values.  (See  unadjusted  length  of  stay  values  presented  in  Table  7.1.) 

Table  E.10  presents  mortality  rates  pre-  and  post-PPS  implementation  without  adjust- 
ment for  sickness  at  admission.  Inhospital  mortality  is  2.3  percentage  points  lower  post-PPS 
(p  <  0.01),  and  mortality  within  30  days  postadmission  is  16.0  percent  in  both  time  periods. 
Mortality  rates  within  six  months  postadmission  were  somewhat  but  insignificantly  higher 
post-PPS  (28.9  percent  pre-PPS  and  29.8  percent  post-PPS).  (See  adjusted  mortality  values 
presented  in  Table  7.2.) 

Table  E.ll  presents  mortality  rates  pre-  and  post-PPS  implementation  with  adjustment 
for  sickness  at  admission.  The  t-values  comparing  mortality  rates  for  the  two  time  periods 
are  listed. 

Table  E.12  shows  the  frequency  with  which  patients  with  a  preadmission  residence  of 
home  or  nursing  home  have  various  discharge  destinations  in  the  pre-  and  post-PPS  periods 
unadjusted  for  sickness  at  admission.  Fewer  patients  with  a  preadmission  residence  of  home 
are  discharged  to  home  in  the  post-PPS  period  (77  percent  pre-PPS  and  73  percent  post-PPS). 
(See  Table  7.3.) 

Table  E.13  shows  the  frequency  with  which  patients  with  a  preadmission  residence  of 
home  or  nursing  home  have  various  discharge  destinations  in  the  pre-  and  post-PPS  periods 
adjusted  for  sickness  at  admission.  The  t-test  is  used  for  comparing  pre-  and  post-PPS  rates 
of  distribution  of  patients  from  home  to  each  discharge  destination  individually. 

Table  E.14  again  compares  discharge  destinations  pre-  and  post-PPS.  However,  for  this 
appendix,  the  chi-square  test  compares  the  distributions  of  patients  admitted  from  home  to 
the  various  discharge  destinations. 

STUDYING  PROLONGED  NURSING  HOME  STAYS 

The  only  available  data  source  for  studying  duration  of  stay  in  a  nursing  home  is 
Medicare's  Part  B  files  of  physician  bills.  When  a  physician  bills  for  a  nursing  home  visit, 
either  the  place  of  service  is  designated  as  a  nursing  home  or  a  special  procedure  code  is 
used.  We  requested  Medicare  Part  B  claims  data  for  patients  from  the  six  insurance  carriers 
responsible  for  such  data  in  the  five  study  states.  Because  of  the  difficulty  and/or  experience 
of  retrieving  the  pre-PPS  data,  we  limited  our  work  with  prolonged  nursing  home  stays  to 
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Table  E.8 

RATES  OF  DISCHARGE   TO  HOME  PRE-  AND  POST-PPS   FOR  HIP   FRACTURE  PATIENTS 
WITH  AND  WITHOUT  ADJUSTMENT  FOR  SICKNESS  AT  ADMISSION  (1) 


Without  With 
Adjustment  Adjustment 


Patients 

Disease  PPS  Period  Discharged  to  Home  (%) 


Hip  Fracture  Pre-PPS  55.7  55.7 

Post-PPS  48.3  48.7 


(1)   Using  scales  designed  to  predict  discharge  to  home.     The  sample 
includes  only  patients  with  a  preadmission  residence  of  home. 
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Table  E.9 
MEAN  LENGTH  OF  STAY 
Pre-  and  Post-PPS(  1  ) 
Adjusted  Mean  LOS  in  Days 


D i  sease 

n 

Pre- 

PPS 

Post 

-PPS 

D  i  f  f  e  rence 

CHF 

2824 

1 1 

.  1 

8 

.8 

-2.3* 

AM  I 

2853 

12 

.7 

10 

.4 

-2.3* 

PNE 

2749 

12 

.  2 

10 

.4 

-1.7* 

CVA 

2824 

16 

.2 

1 1 

.  1 

-5.1* 

H  I  P 

2762 

20 

.2 

14 

.4 

-5.6* 

5  Diseases 

14012 

14 

.4 

1 1 

.00 

-3.4 

*  p  <0.05  for 

compa  r  i  son 

of   length  of 

stay  pre 

-PPS  vs. 

post-PPS 

(1)  Length  of  stay  adjustments  are  made  using  scales  designed 
to  predict  death  at  30  days  post-admission. 


We  used  the  medical    record  as  the  data  base  for  measuring   length  of  stay 
for  each  patient.     We  considered  the  day  of  discharge  from  the  acute 
care  hospitalization  the  day  the  discharge  summary  noted  as  the 
discharge  date.      If  the  patient  ended  the  acute  care  hospitalization  but 
remained   in  the  same  hospital,  ward,   or  bed  (i.e.   a  "paper  discharge") 
to  receive  other  services  (e.g.    rehabilitation  or  skilled  nursing  care), 
we  considered  the  day  of  discharge  the  day  of  termination  of  the  acute 
hosp  i  ta I  stay. 
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Table  E. 14 

USING  MEDICARE  PART  B  BILLS  TO  STUDY  PROLONGED  POST-ADMISSION  NURSING  HOME 
STAY  BY  DISEASE  FOR  PATIENTS  HOME  PRIOR  TO  ADMISSION 


Patients  with  MD  bills  submitted  within 
months  5,   6,  or  7  after  admission  (%)(1) 


Type  of 
Bill(2) 

State 

CHF 

AMI 

PNE 

CVA 

HIP 

DEP 

5  Diseases 

|   6  Diseases 

n-269 

n-280 

n-240 

n-275 

n-302 

n-477 

n=1366 

n=1843 

SN 

A 

3 

2 

4m 

10 

9 

2  j 

6 

5 

SN  or  NH 

A 

5 

2 

6 

14 

12 

4 

8 

7 

*SN/NH/Home  A 

C  0 

"X 

■J  V 

J.  i  | 

O  v 

Office 

A 

£.  0 

£  O 

1  fl 
X  O 

&  n  1 

4  \J 

->  ri 
J  J 

*Home  or 
office 

A 

56 

57 

48 

48 

52 

48 

52 

51 

Any 

A 

82 

75 

68 

74 

72 

73 

74 

74 

n=2  60 

n=267 

n=236 

n=265 

n=263 

n=468  | 

n=1291 

n=1759 

SN  or  NH 

B 

5 

1 

4 

18 

11 

4  1 

8 

7 

n=260 

n=269 

n=259 

n=250 

n=271 

n=428  | 

n=1309 

n=1737 

SN 

C 

2 

1 

2 

5 

2 

2  1 

2 

2 

SN  or  NH/ 
Home 

C 

23 

12 

15 

28 

35 

12  | 

23 

20 

Office 

C 

31 

44 

34 

33 

21 

45 

32 

36 

Any 

C 

69 

70 

57 

70 

64 

69 

66 

66 

*  p  <  0. 
(1)  The 
ted  within 

05 

entries 
months  5 

in  this 
,  6,  or 

table  represent  (the 
7  after  admission/the 

number 
number 

of  patients  with  MD 
of  patients  alive 

bills  submit- 
as  of  7  months 

post-admission)  x  100. 

(2)  Type  of  bill  submitted  within  months  5,  6,  or  7  post-admission. 


330 


three  states.  In  the  following  pages,  we  present  a  series  of  tables  that  support  our  design 
decisions  for  this  section  of  the  study.  In  addition,  we  show  state-specific  and  disease-specific 
data  about  duration  of  nursing  home  stay  following  acute  care  hospitalization. 

Tables  E.15  through  E.17  show  the  fraction  of  patients  with  Medicare  Part  B  physician 
bills  submitted  within  three  months  after  acute  care  hospitalization.  The  tables  present 
disease-specific  data,  data  for  five  study  diseases  (omitting  depression),  and  data  for  six  study 
diseases  (including  depression)  for  the  three  study  states. 

Table  E.15  lists  the  data  for  a  variety  of  patient  types  and  bill  types.  These  data  com- 
bine the  pre-  and  post-PPS  samples. 

Table  E.16  shows  the  fraction  of  patients  with  any  type  of  physician  bill  submitted  for 
various  patient  types.  This  table  presents  data  for  the  pre-  and  post-PPS  samples  combined. 

Table  E.17  shows  the  fraction  of  patients  with  nursing  home  bills  by  patient  type.  In 
this  table  nursing  home  bills  are  denned  as  bills  from  either  a  skilled  nursing  facility,  skilled 
or  other  (residential)  nursing  home  without  distinction,  or  skilled  or  other  nursing  home  or 
home  care  without  distinction.  These  data  are  presented  for  the  pre-  and  post-PPS  sample 
combined. 

Table  E.18  shows  the  fraction  of  patients  with  nursing  home  bills  for  five  diseases  com- 
bined (omitting  depression)  for  the  pre-  compared  with  post-PPS  samples.  This  appendix 
shows  that  the  likelihood  of  having  Medicare  bills  post  discharge  for  patients  going  to  a  nurs- 
ing home  pre-PPS  compared  with  post-PPS  varies  by  state.  There  is  no  significant  difference 
in  the  availability  of  data  in  two  states  although  data  are  more  available  post-PPS  in  State  B. 

Table  E.19  shows  the  fraction  of  patients  with  nursing  home  bills  for  six  diseases  com- 
bined (including  depression)  for  the  pre-  compared  with  post-PPS  samples  although  data  are 
more  available  post-PPS  in  State  B. 

Table  E.20  shows  the  fraction  of  patients  with  physician  bills  submitted  within  months 
5,  6,  or  7  after  admission  by  disease  and  across  disease  (excluding  and  then  including  depres- 
sion patients)  for  the  pre-  and  post-PPS  samples  combined.  The  data  are  presented  for  vari- 
ous definitions  of  nursing  home  bills. 

Table  E.21  shows  the  fraction  of  patients  with  physician  bills  submitted  within  months 
5,  6,  or  7  after  admission  by  disease  and  across  disease  (excluding  and  then  including  depres- 
sion patients)  for  the  pre-PPS  compared  with  the  post-PPS  periods. 

Tables  E.15  through  E.19  show  that  most  (>  85  percent)  patients  for  whom  we  suspect 
bills  should  be  submitted  do  have  Medicare  Part  B  bills  on  file. 

Table  E.21  shows  a  trend  toward  more  patients  staying  in  nursing  homes  months  5,  6, 
or  7  after  acute  hospitalization  in  the  post-PPS  period  than  the  pre-PPS  period.  The  pre-post 
difference  is  significant  for  only  one  of  the  three  states  studied. 
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